Biography:Bernard Widrow

From HandWiki
Bernard Widrow
Widrow with Adaline.svg
Widrow demonstrating the "Knobby Adaline" device (1963)
Born (1929-12-24) December 24, 1929 (age 94)
NationalityUnited States
Alma materMassachusetts Institute of Technology[1]
Scientific career
FieldsElectrical engineering
InstitutionsStanford University
Doctoral advisorWilliam Linvill
Doctoral students

Bernard Widrow (born December 24, 1929) is a U.S. professor of electrical engineering at Stanford University.[1] He is the co-inventor of the Widrow–Hoff least mean squares filter (LMS) adaptive algorithm with his then doctoral student Ted Hoff.[2] The LMS algorithm led to the ADALINE and MADALINE artificial neural networks and to the backpropagation technique. He made other fundamental contributions to the development of signal processing in the fields of geophysics, adaptive antennas, and adaptive filtering. A summary of his work is.[3]

He is the namesake of "Uncle Bernie's Rule": the training sample size should be 10 times the number of weights in a network.[4][5]

Biography

This section is based on.[6][7]

Early life and education

He was born in Norwich, Connecticut. While young, he was interested in electronics. During WWII, he found an entry on "Radios" in the World Book Encyclopedia, and built a one-tube radio.

He entered MIT in 1947, studied electrical engineering and electronics, and graduated in 1951. After that, he got a research assistantship in the MIT Digital Computer Laboratory, in the magnetic core memory group. The DCL was a division of the Servomechanisms Laboratory,[8] which was building the Whirlwind I computer. The experience of building magnetic core memory shaped his understanding of computers into a "memory's eye view", that is, he "look for the memory and see what you have to connect around it".

For his masters thesis (1953, advised by William Linvill), he worked on raising the signal-to-noise ratio of the sensing signal of magnetic core memory. Back then, the hysteresis loops for magnetic core memory was not square enough, making sensing signal noisy.

For his PhD (1956, advised by William Linvill), he worked on the statistical theory of quantization noise,[9] inspired by work by William Linvill and David Middleton.[10]

During PhD, he learned the Wiener filter from Lee Yuk-wing. To design a Wiener filter, one must know the statistics of the noiseless signal that one wants to recover. However, if the statistics of the noiseless signal is unknown, this cannot be designed. Widrow thus designed an adaptive filter that uses gradient descent to minimize the mean square error. He also attended the Dartmouth workshop in 1956 and was inspired to work on AI.

Work on AI

In 1959, he got his first graduate student, Ted Hoff. They improved the previous adaptive filter so that it makes a gradient descent for each datapoint, resulting in the delta rule and the ADALINE. To avoid having to hand-tune the weights in ADALINE, they invented the memistor, with conductance (ADALINE weights) being the thickness of the copper on the graphite.

During a meeting with Frank Rosenblatt, Widrow argued that the S-units in the perceptron machine should not be connected randomly to the A-units. Instead, the S-units should be removed, so that the photocell inputs would be directly inputted into the A-units. Rosenblatt objected that "the human retina is built that way".

Despite many attempts, they never succeeded in developing a training algorithm for a multilayered neural network. The furthest they got was with Madaline Rule I (1962), which had two weight layers. The first was trainable, but the second was fixed. Widrow stated their problem would have been solved by the backpropagation algorithm. "This was long before Paul Werbos. Backprop to me is almost miraculous."

Adaptive filtering

Unable to train multilayered neural networks, Widrow turned to adaptive filtering and adaptive signal processing, using techniques based on the LMS filter for applications such as adaptive antenna,[11] adaptive noise canceling,[12] and applications to medicine.[13]

At a 1985 conference in Snowbird, Utah, he noticed that neural network research was returning, and he also learned of the backpropagation algorithm. After that, he returned to neural network research.

Publications

  • 1965 "A critical comparison of two kinds of adaptive classification networks", K. Steinbuch and B. Widrow, IEEE Transactions on Electronic Computers, pp. 737–740.
  • 1985 B. Widrow and S. D. Stearns. Adaptive Signal Processing. New Jersey: Prentice-Hall, Inc., 1985.
  • 1994 B. Widrow and E. Walach. Adaptive Inverse Control. New Jersey: Prentice-Hall, Inc., 1994.
  • 2008 B. Widrow and I. Kollar. Quantization Noise: Roundoff Error in Digital Computation, Signal Processing, Control, and Communications. Cambridge University Press, 2008.

Honors

  • Elected Fellow IEEE, 1976[2]
  • Elected Fellow AAAS, 1980[2]
  • IEEE Centennial Medal, 1984[2]
  • IEEE Alexander Graham Bell Medal, 1986[2]
  • IEEE Neural Networks Pioneer Medal, 1991[2]
  • Inducted into the National Academy of Engineering, 1995
  • IEEE Signal Processing Society Award, 1999
  • IEEE Millennium Medal, 2000
  • Benjamin Franklin Medal, 2001[14]
  • International Neural Network Society (INNIS) Board member 2004

He was one of the Board of Governors of the International Neural Network Society (INNIS) in 2003.

References

  1. ↑ 1.0 1.1 "Widrow's Stanford web page". Information Systems Laboratory, Electrical Engineering Department, Stanford University. http://www-isl.stanford.edu/~widrow/. 
  2. ↑ 2.0 2.1 2.2 2.3 2.4 2.5 Andrew Goldstein (1997). "Bernard Widrow Oral History". IEEE Global History Network. IEEE. http://www.ieeeghn.org/wiki/index.php/Oral-History:Bernard_Widrow. Retrieved 22 August 2011. 
  3. ↑ Widrow, B.; Lehr, M.A. (September 1990). "30 years of adaptive neural networks: perceptron, Madaline, and backpropagation". Proceedings of the IEEE 78 (9): 1415–1442. doi:10.1109/5.58323. http://ieeexplore.ieee.org/document/58323/. 
  4. ↑ Morgan, N.; Bourlard, H. (1989). "Generalization and Parameter Estimation in Feedforward Nets: Some Experiments". Advances in Neural Information Processing Systems (Morgan-Kaufmann) 2. https://proceedings.neurips.cc/paper_files/paper/1989/hash/63923f49e5241343aa7acb6a06a751e7-Abstract.html. 
  5. ↑ "(1960) Bernard Widrow and Marcian E. Hoff, "Adaptive switching circuits," [i1960 IRE WESCON Convention Record[/i], New York: IRE, pp. 96-104."], Neurocomputing, Volume 1 (The MIT Press): pp. 123–134, 1988-04-07, doi:10.7551/mitpress/4943.003.0012, ISBN 9780262267137, http://dx.doi.org/10.7551/mitpress/4943.003.0012, retrieved 2023-11-03 
  6. ↑ "Bernard Widrow, an oral history conducted in 1997 by Andrew Goldstein, IEEE History Center, Piscataway, NJ, USA" (in en). 1997. https://ethw.org/Oral-History:Bernard_Widrow. 
  7. ↑ Anderson, James A., ed (2000) (in en). Talking Nets: An Oral History of Neural Networks. The MIT Press. doi:10.7551/mitpress/6626.003.0004. ISBN 978-0-262-26715-1. https://direct.mit.edu/books/book/4886/Talking-NetsAn-Oral-History-of-Neural-Networks. 
  8. ↑ "Collection: Massachusetts Institute of Technology, Digital Computer Laboratory records | MIT ArchivesSpace". https://archivesspace.mit.edu/repositories/2/resources/311. 
  9. ↑ Widrow, B. (1956). "A Study of Rough Amplitude Quantization by Means of Nyquist Sampling Theory". IRE Transactions on Circuit Theory 3 (4): 266–276. doi:10.1109/TCT.1956.1086334. ISSN 0096-2007. http://ieeexplore.ieee.org/document/1086334/. 
  10. ↑ "Oral-History:David Middleton (2000)" (in en). 2021-01-26. https://ethw.org/Oral-History:David_Middleton_(2000). 
  11. ↑ Widrow, B.; Mantey, P.E.; Griffiths, L.J.; Goode, B.B. (1967). "Adaptive antenna systems". Proceedings of the IEEE 55 (12): 2143–2159. doi:10.1109/PROC.1967.6092. ISSN 0018-9219. http://ieeexplore.ieee.org/document/1448022/. 
  12. ↑ Widrow, B.; Glover, J.R.; McCool, J.M.; Kaunitz, J.; Williams, C.S.; Hearn, R.H.; Zeidler, J.R.; Eugene Dong, Jr. et al. (1975). "Adaptive noise cancelling: Principles and applications". Proceedings of the IEEE 63 (12): 1692–1716. doi:10.1109/PROC.1975.10036. ISSN 0018-9219. https://ieeexplore.ieee.org/document/1451965. 
  13. ↑ Yelderman, Mark; Widrow, Bernard; Cioffi, John M.; Hesler, Edward; Leddy, Jeffrey A. (July 1983). "ECG Enhancement by Adaptive Cancellation of Electrosurgical Interference". IEEE Transactions on Biomedical Engineering BME-30 (7): 392–398. doi:10.1109/TBME.1983.325039. ISSN 0018-9294. http://ieeexplore.ieee.org/document/4121661/. 
  14. ↑ Abend, Kenneth (2002). "The 2001 Benjamin Franklin Medal in Engineering presented to Bernard Widrow - Journal of the Franklin Institute - Tom 339, Numer 3 (2002) - Biblioteka Nauki - Yadda". Journal of the Franklin Institute 3 (339): 283–294. doi:10.1016/S0016-0032(01)00044-8. http://yadda.icm.edu.pl/yadda/element/bwmeta1.element.elsevier-cd322222-a407-32f6-a89c-4d7de1137068. 
Awards
Preceded by
Charles K. Kao 		IEEE Alexander Graham Bell Medal
1986 		Succeeded by
Joel S. Engel, Richard H. Frenkiel and William C. Jakes, Jr.



Retrieved from "https://handwiki.org/wiki/index.php?title=Biography:Bernard_Widrow&oldid=3399863"