Publications

2023

1. 

   When Layers Play the Lottery, all Tickets Win at Initialization

   Artur Jordao, George Araújo, Helena Almeida Maia, and Helio Pedrini

   In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops , Oct 2023

   TLDR Abs arXiv Bib HTML PDF Supp

   Pruning is a standard technique for reducing the computational cost of deep networks. Many advances in pruning leverage concepts from the Lottery Ticket Hypothesis (LTH). LTH reveals that inside a trained dense network exists sparse subnetworks (tickets) able to achieve similar accuracy (i.e., win the lottery - winning tickets). Pruning at initialization focuses on finding winning tickets without training a dense network. Studies on these concepts share the trend that subnetworks come from weight or filter pruning. In this work, we investigate LTH and pruning at initialization from the lens of layer pruning. First, we confirm the existence of winning tickets when the pruning process removes layers. Leveraged by this observation, we propose to discover these winning tickets at initialization, eliminating the requirement of heavy computational resources for training the initial (over-parameterized) dense network. Extensive experiments show that our winning tickets notably speed up the training phase and reduce up to 51% of carbon emission, an important step towards democratization and green Artificial Intelligence. Beyond computational benefits, our winning tickets exhibit robustness against adversarial and out-of-distribution examples. Finally, we show that our subnetworks easily win the lottery at initialization while tickets from filter removal (the standard structured LTH) hardly become winning tickets.

   Our subnetworks easily win the lottery at initialization, speed up the training, and exhibit robustness against adversarial and out-of-distribution examples.

   @inproceedings{Jordao_2023_ICCV,
     author = {Jordao, Artur and de Araújo, George and de Almeida Maia, Helena and Pedrini, Helio},
     booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops},
     month = oct,
     title = {When Layers Play the Lottery, all Tickets Win at Initialization},
     year = {2023}
   }

2. 

   Single Image Super-Resolution Based on Capsule Neural Networks

   George Corrêa Araújo, Artur Jordão, and Helio Pedrini

   In Brazilian Conference on Intelligent Systems , Oct 2023

   TLDR Abs arXiv Bib HTML Slides

   Single image super-resolution (SISR) consists of obtaining one high-resolution version of a low-resolution image by increasing the number of pixels per unit area. This method has been actively investigated by the research community, due to the wide variety of problems ranging from real-world surveillance to aerial and satellite imaging. Most of the improvements in SISR come from convolutional networks, in which approaches often focus on the deeper and wider architectural paradigm. In this work, we decided to step up from the traditional convolutions and adopt the concept of capsules. Since their overwhelming results in image classification and segmentation problems, we question how suitable they are for SISR. We also verify that different solutions share similar configurations, and argue that this trend leads to fewer explorations of network designs. Throughout our experiments, we check various strategies to improve results, ranging from new and different loss functions to changes in the capsule layers. Our network achieved positive and promising results with fewer convolutional-based layers, showing that capsules might be a concept worth applying to the image super-resolution problem. In particular, we observe that the proposed method recreates the connection between the different characters more precisely, thus demonstrating the potential of capsules in super-resolution problems.

   Our network achieved good results with fewer convolutional-based layers, showing that capsules might be a concept worth applying in the SISR problem.

   @inproceedings{10.1007/978-3-031-45392-2_8,
     author = {de Araújo, George Corrêa and Jordão, Artur and Pedrini, Helio},
     booktitle = {Brazilian Conference on Intelligent Systems},
     doi = {10.1007/978-3-031-45392-2_8},
     editor = {Naldi, Murilo C. and Bianchi, Reinaldo A. C.},
     isbn = {978-3-031-45392-2},
     publisher = {Springer Nature Switzerland},
     title = {Single Image Super-Resolution Based on Capsule Neural Networks},
     year = {2023}
   }