“Mad Max: Affine Spline Insights into Deep Learning"
Frontiers of Deep Learning Workshop, Simons Institute
16 July 2019


References:

Co-authors:  Randall Balestriero, Jack Wang, Hamid Javadi

An alternative presentation at the Alan Turing Institute, May 2019 (Get your SPARFA merchandise here!)

DSP group members will be traveling en masse to New Orleans in May 2019 to present four regular papers at the International Conference on Learning Representations

Two workshops have been accepted for NIPS in December 2018; more details soon on how to contribute:

  • Integration of Deep Learning Theories (R. G. Baraniuk, S. Mallat,  A. Anandkumar, A. Patel, and N. Ho)
  • Machine Learning for Geophysical & Geochemical Signals (L. Pyrak-Nolte, J. Morris, J. Rustad, R. G. Baraniuk)

This year, over 2.2 million students are saving an estimated $177 million by using free textbooks from OpenStax, the Rice University-based publisher of open educational resource materials.  Since 2012, OpenStax's 29 free, peer-reviewed, openly licensed textbooks for the highest-enrolled high school and college courses have been used by more than 6 million students. This year, OpenStax added several new books to its library, including Biology for AP Courses, Introductory Business Statistics and second editions of its economics titles.

OpenStax books are having a tangible, marketwide impact, according to a 2017 Babson Survey that found that “the rate of adoption of OpenStax textbooks among faculty teaching large-enrollment courses is now at 16.5%, a rate which rivals that of most commercial textbooks."  "We're excited about the rapidly growing number of instructors making the leap to open textbooks," said OpenStax founder Richard Baraniuk, the Victor E. Cameron Professor of Electrical and Computer Engineering at Rice. "Our community is creating a movement that will make a big impact on college affordability. The success of open textbooks like OpenStax have ignited competition in the textbook market, and textbook prices are actually falling for the first time in 50 years."

Read the full press release

C. A. Metzler, A. Mousavi, R. Heckel, and R. G. Baraniuk, “Unsupervised Learning with Stein’s Unbiased Risk Estimator,” https://arxiv.org/abs/1805.10531, June 2018.

Learning from unlabeled and noisy data is one of the grand challenges of machine learning. As such, it has seen a flurry of research with new ideas proposed continuously. In this work, we revisit a classical idea: Stein's Unbiased Risk Estimator (SURE). We show that, in the context of image recovery, SURE and its generalizations can be used to train convolutional neural networks (CNNs) for a range of image denoising and recovery problems without any ground truth data.  Specifically, our goal is to  reconstruct an image x from a noisy linear transformation (measurement) of the image. We consider two scenarios: one where no additional data is available and one where we have measurements of other images that are drawn from the same noisy distribution as x, but have no access to the clean images. Such is the case, for instance, in the context of medical imaging, microscopy, and astronomy, where noise-less ground truth data is rarely available.  We show that in this situation, SURE can be used to estimate the mean-squared-error loss associated with an estimate of x. Using this estimate of the loss, we train networks to perform denoising and compressed sensing recovery. In addition, we also use the SURE framework to partially explain and improve upon an intriguing results presented by Ulyanov et al. in "Deep Image Prior": that a network initialized with random weights and fit to a single noisy image can effectively denoise that image.
 

Appearing today on NSF's Science360 News:  Taking a closer look with a lens-free fluorescent microscope

Rice University engineers are building a flat microscope, called FlatScope (TM), and developing software that can decode and trigger neurons on the surface of the brain.  The goal as part of a new government initiative is to provide an alternate path for sight and sound to be delivered directly to the brain.  The project is part of a $65 million effort announced this week by the federal Defense Advanced Research Projects Agency (DARPA) to develop a high-resolution neural interface. Among many long-term goals, the Neural Engineering System Design (NESD) program hopes to compensate for a person's loss of vision or hearing by delivering digital information directly to parts of the brain that can process it.

More info