Ключевые слова
искусственные нейронные сети
кроссбар
модель Хопфилда
Как цитировать
Аннотация
исследованы свойства нейронной сети в случае, когда ранг тензора связей выше двух, т.е. помимо матрицы синаптических связей присутствуют пресинаптические синапсы. Такого рода тензоры связи имеют место при аппаратной реализации нейронной сети на основе кроссбаров. Неотъемлемым свойством кроссбарной схемы является наличие паразитных токов: сигнал с одной шины, идущей на некоторый нейрон, перетекает через ячейки памяти (синапсы) на шины всех других нейронов. Тем самым на входной сигнал нейрона накладывается шум — ослабленные сигналы, идущие на все остальные нейроны. При этом проводимость аналоговой кроссбарной ячейки изменяется пропорционально шумовому сигналу и выходной сигнал ячейки приобретает нелинейный по входу характер. Показано, что при определенном виде тензора связей качество нейросетевого алгоритма значительно улучшается. Анализ проведен на примере сети, подобной сети Хопфилда.
Литература
Pershin Y., Di Ventra M. Experimental Demonstration of Associative Memory with Memristive Neural Networks. Neural Networks. 2010;23(7):881–886.
Jo S. H., Chang T., Ebong I. et al. Nanoscale Memristor Device as Synapse in Neuromorphic Systems. Nanoletters. 2010;10(4):1297–1301.
Truong S. N., Ham S. J., Min K. S. Neuromorphic Crossbar Circuit with Nanoscale FilamentarySwitching Binary Memristors for Speech Recognition. Nanoscale Res. Lett. 2014;9:629–629.
Zhu X., Yang X., Wu C. et al. Hamming Network Circuits Based on CMOS/Memristor Hybrid Design. IEICE Electronics Express. 2013;10(12):20130404-20130404.
Palagushkin A. N., Roshchupkin D. V., Yudkin F. A. et al. Aspects of the a-TiOx Memristor Active Medium Technology. J. Appl. Phys. 2018;124(20):205109.
Морозов А. Ю., Ревизников Д. Л., Абгарян К. К. Известия вузов. Вопросы реализации нейросетевых алгоритмов на мемристорных кроссбарах. Материалы электронной техники. 2019;22(4):272– 278.
Kim H., Mahmoodi M. R., Nili H., Strukov D. B. 4K-memristor Analog-Grade Passive Crossbar Circuit. Nat. Commun. 2021;12(1):5198.
Liu X., Zeng Z. Memristor Crossbar Architectures for Implementing Deep Neural Networks. Complex & Intelligent System. 2022;8:787–802.
Kaneko Y., Nishitani Y., Ueda M. Ferroelectric Artificial Synapses for Recognition of a Multishaded Image. IEEE Trans. Electron Devices. 2014;61:2827–2833.
Boyn S. et al. Learning Through Ferroelectric Domain Dynamics in Solid-State Synapses. Nat. Commun. 2017;8(1):14736.
Romera M. et al. Vowel Recognition with Four Coupled Spin-Torque Nanooscillators. Nature. 2018;563:230–234.
Fuller E. J. et al. Parallel Programming of an Ionic Floating-Gate Memory Array for Scalable Neuromorphic Computing. Science. 2019;364:570–574.
Goswami S. et al. Robust Resistive Memory Devices Using Solution-Processable Metal-Coordinated Azo Aromatics. Nat. Mater. 2017;16:1216–1224.
Indiveri G. et al. Integration of Nanoscale Memristor Synapses in Neuromorphic Computing Architectures. Nanotechnology. 2013;24:384010.
Prezioso M. et al. Training and Operation of an Integrated Neuromorphic Network Based on Metal-Oxide Memristors. Nature. 2015;521:61–64.
Ambrogio S. et al. Neuromorphic Learning and Recognition with One-Transistor-One-Resistor Synapses and Bistable Metal Oxide RRAM. IEEE Trans. Electron Devices. 2016;63:1508–1515.
Adam G. C. et al. 3-D Memristor Crossbars for Analog and Neuromorphic Computing Applications. IEEE Trans. Electron Devices. 2017;64:312–318.
Hu M. et al. Memristor-Based Analog Computation and Neural Network Classification with a Dot Product Engine. Adv. Mater. 2018;30:1705914.
Merrikh B. F. et al. Implementation of Multilayer Perceptron Network with Highly Uniform Passive Memristive Crossbar Circuits. Nat. Commun. 2018;9(1).
Chakrabartty S., Cauwenberghs G. Sub-Microwatt Analog VLSI Trainable Pattern Classifier. IEEE J. Solid-State Circuits. 2007;42:1169–1179.
Ramakrishnan S., Hasler J. Vector-Matrix Multiply and Winner-Take-All as an Analog Classifier. IEEE Trans. Very Large Scale Integr. Syst. 2014;22:353–361.
Merrikh B. F. et al. High-Performance Mixed-Signal Neurocomputing with Nanoscale Floating-Gate Memory Cells. IEEE Trans. Neural Netw. Learn. Syst. 2018;29:4782–4790.
Eryilmaz S. B. et al. Brain-Like Associative Learning Using a Nanoscale Nonvolatile Phase Change Synaptic Device Array. Front. Neurosci. 2014;8:205.
Burr G. W. et al. Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element. IEEE Trans. Electron Devices. 2015;62:3498–3507.
Boybat I. et al. Neuromorphic Computing with Multi-Memristive Synapses. Nat. Commun. 2018;9(1).
Hopfield J. J. Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proc. Nat. Acad. Sci. USA. 1982;79:2554-2558.
Amit D. J., Gutfreund H., Sompolinsky H. Storing Infinite Numbers of Patterns in a Spin-Glass Model of Neural Networks. Physical Review Letters. 1985;55(14):1530–1533.
Karandashev I., Kryzhanovsky B., Litinskii L. Weighted Patterns as a Tool to Improve the Hopfield Model. Physical Review E. 2012;85:041925.
Kryzhanovsky B. V., Kryzhanovsky V. M., Mikaelian A. L. et al. Parametric Dynamic Neural Network Recognition Power. Optical Memory & Neural Network. 2001;10(4):211-218.
Kryzhanovsky B. V., Mikaelian A. L. On the Recognition Ability of a Neural Network on Neurons with Parametric Transformation of Frequencies. Doklady Mathematics. 2002;65(2):286-288.
Kryzhanovsky B., Litinskii L., Fonarev A. An Effective Associative Memory for Pattern Recognition. Lecture Notes in Computer Science. 2003;2810:179-186.
Kryzhanovsky B. V., Kryzhanovsky V. M. Modified q-state Potts Model with Binarized Synaptic Coefficients. Lecture Notes in Computer Science. 2008;5164:72-80.
Karandashev I., Kryzhanovsky B., Litinskii L. Properties of the Hopfield Model with Weighted Patterns. Lecture Notes in Computer Science. 2012;7552:41-48.
Agliari E., Albanese L., Alemanno F. et al. Dense Hebbian Neural Networks: A Replica Symmetric Picture of Supervised Learning. Physica A: Statistical Mechanics and Its Applications. 2023;626:129076.
Barra A., Bernacchia A., Santucci E., Contucci P. On the Equivalence of Hopfield Networks and Boltzmann Machines. Neural Networks. 2012;34:1–9.
Bertram R., Rubin J. E. Multi-Timescale Systems and Fast-Slow Analysis. Mathematical Biosciences. 2017;287:105–121.
Fanaskov V., Oseledets I. Associative Memory and Dead Neurons. arXiv:2410.13866.
Hinton G. E. A Practical Guide to Training Restricted Boltzmann Machines. Neural Networks: Tricks of the Trade. Springer-Verlag Berlin Heidelberg; 2012:599–619.
Hoover B., Chau D. H., Strobelt H. et al. A Universal Abstraction for Hierarchical Hopfield Networks. The Symbiosis of Deep Learning and Differential Equations II. NeurIPS; 2022.
Krotov D., Hopfield J. Dense Associative Memory is Robust to Adversarial Inputs. Neural Computation. 2018;30(12):3151–3167.
Hebb D. O. The Organization of Behavior. A Neuropsychological Theory. NY: Wiley; 1949.
Экклс Д. К. Физиология нервных клеток. М.: ИЛ; 1959.