Optimization induced equilibrium networks: An explicit optimization perspective for understanding equilibrium models
In this paper, we decompose the feed-forward DNN and find a more reasonable basic unit layer, which shows a close relationship with the proximal operator. Based on it, we propose new equilibrium models, OptEqs, and explore their underlying optimization problems thoroughly. We provide two strategies to introduce customized regularizations to the equilibrium points, and achieve significant performance improvement in experiments. We highlight that by modifying the underlying optimization problems, we can create more effective network architectures. Our work may inspire more interpretable equilibrium models from the optimization perspective.
Jun 10, 2022
Towards efficient scene understanding via squeeze reasoning
Jul 30, 2021
PointFlow: Flowing Semantics Through Points for Aerial Image Segmentation
Feb 13, 2021
Is Attention Better Than Matrix Decomposition?
Self-attention is not better than the matrix decomposition~(MD) model developed 20 years ago regarding the performance and computational cost for encoding the long-distance dependencies.
Sep 28, 2020
Improving Semantic Segmentation via Decoupled Body and Edge Supervision
Jul 3, 2020
Spatial Pyramid Based Graph Reasoning for Semantic Segmentation
Feb 24, 2020
SOGNet: Scene Overlap Graph Network for Panoptic Segmentation
We leverage each object’s category, geometry and appearance features to perform relational embedding, and output a relation matrix that encodes overlap relations. In order to overcome the lack of supervision, we introduce a differentiable module to resolve the overlap between any pair of instances.
Feb 7, 2020
Dynamic System Inspired Adaptive Time Stepping Controller for Residual Networks Families
We analyze the effects of time stepping on the Euler method and ResNets. We establish a stability condition for ResNets with step sizes and weight parameters, and point out the effects of step sizes on the stability and performance. Inspired by our analyses, we develop an adaptive time stepping controller that is dependent on the parameters of the current step, and aware of previous steps.
Feb 7, 2020
Expectation Maximization Attention Networks for Semantic Segmentation
We formulate the attention mechanism into an expectation-maximization manner and iteratively estimate a much more compact set of bases upon which the attention maps are computed.
Jul 22, 2019
R^2 Net Recurrent and Recursive Network for Sparse View CT Artifacts Removal
We propose a novel neural network architecture to reduce streak artifacts generated in sparse-view 2D Cone Beam Computed To-mography (CBCT) image reconstruction.
Jun 19, 2019
Recurrent Squeeze-and-Excitation Net for Single Image Deraining
We propose a novel deep network architecture based on deep convolutional and recurrent neural networksfor single image deraining.
Jul 19, 2018