Evidence
Neural Netw. 2024 Jul 1;179:106499. doi: 10.1016/j.neunet.2024.106499. Online ahead of print.
ABSTRACT
Brain-inspired Spiking Neural Networks (SNNs) have attracted much attention due to their event-based computing and energy-efficient features. However, the spiking all-or-none nature has prevented direct training of SNNs for various applications. The surrogate gradient (SG) algorithm has recently enabled spiking neural networks to shine in neuromorphic hardware. However, introducing surrogate gradients has caused SNNs to lose their original sparsity, thus leading to the potential performance loss. In this paper, we first analyze the current problem of direct training using SGs and then propose Masked Surrogate Gradients (MSGs) to balance the effectiveness of training and the sparseness of the gradient, thereby improving the generalization ability of SNNs. Moreover, we introduce a temporally weighted output (TWO) method to decode the network output, reinforcing the importance of correct timesteps. Extensive experiments on diverse network structures and datasets show that training with MSG and TWO surpasses the SOTA technique.
PMID:39013289 | DOI:10.1016/j.neunet.2024.106499
Estimated reading time: 3 minute(s)
Latest: Psychiatryai.com #RAISR4D Evidence
Cool Evidence: Engaging Young People and Students in Real-World Evidence
Real-Time Evidence Search [Psychiatry]
AI Research
Directly training temporal Spiking Neural Network with sparse surrogate gradient
🌐 90 Days
AI Virtual Reality Related Evidence Matrix
- GABAergic synaptic scaling is triggered by changes in spiking activity rather than AMPA receptor activation
- Protocol-based control for semi-Markov reaction-diffusion neural networks
- Heterogeneous coexisting attractors, large-scale amplitude control and finite-time synchronization of central cyclic memristive neural networks
- Inference of gene regulatory networks based on directed graph convolutional networks
- Global and multi-partition local network analysis of scalp EEG in West syndrome before and after treatment
- Disentangling temporal and rate codes in primate somatosensory cortex
- A robust multi-scale feature extraction framework with dual memory module for multivariate time series anomaly detection
- Bimodal alteration of cognitive accuracy for spintronic artificial neural networks
- Bimodal alteration of cognitive accuracy for spintronic artificial neural networks
- Proton Conducting Neuromorphic Materials and Devices
- Reporting of surrogate endpoints in randomised controlled trial reports (CONSORT-Surrogate): extension checklist with explanation and elaboration
- Reporting of surrogate endpoints in randomised controlled trial protocols (SPIRIT-Surrogate): extension checklist with explanation and elaboration
- Acute Exercise Improves Large-Scale Brain Network Segregation in Healthy Older Adults
- Determinantal point process attention over grid cell code supports out of distribution generalization
- Federated learning using model projection for multi-center disease diagnosis with non-IID data
- Exploring aperiodic activity in first episode schizophrenia spectrum psychosis: A resting-state EEG analysis
- Exploring aperiodic activity in first episode schizophrenia spectrum psychosis: A resting-state EEG analysis
- Persistent Interruption in Parvalbumin Positive Inhibitory Interneurons: Biophysical and Mathematical Mechanisms
- Sharing massive biomedical data at magnitudes lower bandwidth using implicit neural function
- Triple-0: Zero-shot denoising and dereverberation on an end-to-end frozen anechoic speech separation network
- Neural waves and computation in a neural net model II: Data-like structures and the dynamics of episodic memory
- Brain functional gradient and structure features in adolescent and adult autism spectrum disorders
- Conjointly active and passive modelings with deep neural networks as fully automated optimizations for upper-mid band 6G communications
- RevGraphVAMP: A protein molecular simulation analysis model combining graph convolutional neural networks and physical constraints
- Corticotropin releasing factor alters the functional diversity of accumbal cholinergic interneurons
- Exploring visuospatial function neuroplasticity in elite speed skaters: a resting-state fMRI independent component analysis
- An evaluation of recent advancements in biological sensory organ-inspired neuromorphically tuned biomimetic devices
- An evaluation of recent advancements in biological sensory organ-inspired neuromorphically tuned biomimetic devices
- Potential Role of Generative Adversarial Networks in Enhancing Brain Tumors
- SpaDE: Semantic Locality Preserving Biclustering for Neuroimaging Data
- SpaDE: Semantic Locality Preserving Biclustering for Neuroimaging Data
- Deep learning with convolution neural network detecting mesiodens on panoramic radiographs: comparing four models
- LCGNet: Local Sequential Feature Coupling Global Representation Learning for Functional Connectivity Network Analysis with fMRI
- Efficient Mixed-Precision Matrix Factorization of the Inverse Overlap Matrix in Electronic Structure Calculations with AI-Hardware and GPUs
- LOCUS: A REGULARIZED BLIND SOURCE SEPARATION METHOD WITH LOW-RANK STRUCTURE FOR INVESTIGATING BRAIN CONNECTIVITY
- A novel diagnosis method combined dual-channel SE-ResNet with expert features for inter-patient heartbeat classification
- A Comprehensive Review on Deep Learning Techniques in Alzheimer's Disease Diagnosis
- Shared and Distinctive Brain Networks Underlying Trait and State Rumination
- Navigating the frontier of drug-like chemical space with cutting-edge generative AI models
- Complex artificial intelligence models for energy sustainability in educational buildings
- Children's oppositional defiant disorder symptoms and neural synchrony in mother-child interactions: An fNIRS study
- Predictive Power of a Bayesian Effective Action for Fully Connected One Hidden Layer Neural Networks in the Proportional Limit
- Children's oppositional defiant disorder symptoms and neural synchrony in mother-child interactions: An fNIRS study
- Neural correlates of response to joint attention in 2-to-5-year-olds in relation to ASD and social-communicative abilities: An fNIRS and behavioral study
- Variable data structures and customized deep learning surrogates for computationally efficient and reliable characterization of buried objects
- Identifying behavioral links to neural dynamics of multifiber photometry recordings in a mouse social behavior network
- Dynamic formation of a posterior-to-anterior peak-alpha-frequency gradient driven by two distinct processes
- Sequential safe static and dynamic screening rule for accelerating support tensor machine
- Intrinsic neural timescales exhibit different lengths in distinct meditation techniques
- An efficient method for disaster tweets classification using gradient-based optimized convolutional neural networks with BERT embeddings
- Degraded tactile coding in the Cntnap2 mouse model of autism
- Deciphering social traits and pathophysiological conditions from natural behaviors in common marmosets
- Exploring functional and structural connectivity disruptions in spinocerebellar ataxia type 3: Insights from gradient analysis
- Representations and generalization in artificial and brain neural networks
- No evidence of musical training influencing the corti-cal contribution to the speech-FFR and its modulation through selective attention
- Automated diagnosis of schizophrenia based on spatial-temporal residual graph convolutional network
- Protein ligand binding site prediction using graph transformer neural network
- Artificial Intelligence in Parkinson's Disease: Early Detection and Diagnostic Advancements
- Research on bearing fault diagnosis based on improved genetic algorithm and BP neural network
- Histological tissue classification with a novel statistical filter-based convolutional neural network
- Histological tissue classification with a novel statistical filter-based convolutional neural network
- Investigating quantitative approach for microalgal biomass using deep convolutional neural networks and image recognition
- Tau follows principal axes of functional and structural brain organization in Alzheimer's disease
- Prediction of dose distributions for non-small cell lung cancer patients using MHA-ResUNet
Evidence Blueprint
Directly training temporal Spiking Neural Network with sparse surrogate gradient
☊ AI-Driven Related Evidence Nodes
(recent articles with at least 5 words in title)
More Evidence
Directly training temporal Spiking Neural Network with sparse surrogate gradient
🌐 365 Days
AI Virtual Reality Related Evidence Matrix
- Spike Attention Coding for Spiking Neural Networks
- Sparse Spiking Neural-Like Membrane Systems on Graphics Processing Units
- GABAergic synaptic scaling is triggered by changes in spiking activity rather than AMPA receptor activation
- Protocol-based control for semi-Markov reaction-diffusion neural networks
- Reconstructing computational system dynamics from neural data with recurrent neural networks
- Predicting Blood Glucose Levels with Organic Neuromorphic Micro-Networks
- Distributed adaptive robust containment control for reaction-diffusion neural networks with external disturbances under directed graphs
- A fully spiking coupled model of a deep neural network and a recurrent attractor explains dynamics of decision making in an object recognition task
- DeforT: Deformable transformer for visual tracking
- Brain topology improved spiking neural network for efficient reinforcement learning of continuous control
- HybMED: A Hybrid Neural Network Training Processor with Multi-Sparsity Exploitation for Internet of Medical Things
- Towards complex dynamic physics system simulation with graph neural ordinary equations
- Connectome-based reservoir computing with the conn2res toolbox
- Heterogeneous coexisting attractors, large-scale amplitude control and finite-time synchronization of central cyclic memristive neural networks
- Inference of gene regulatory networks based on directed graph convolutional networks
- Intrinsic and synaptic contributions to repetitive spiking in dentate granule cells
- Spintronics-Based Neuromorphic and Ising Computing
- Stochastic neuro-fuzzy system implemented in memristor crossbar arrays
- Deep Photonic Reservoir Computer for Speech Recognition
- Biophysical modulation and robustness of itinerant complexity in neuronal networks
- Gamma and Beta Band Oscillation in Working Memory given Sequential or Concurrent Multiple Items: A Spiking Network Model
- Global and multi-partition local network analysis of scalp EEG in West syndrome before and after treatment
- A method for estimating dynamic functional network connectivity gradients (dFNG) from ICA captures smooth inter-network modulation
- Imp is expressed in INPs and newborn neurons where it regulates neuropil targeting in the central complex
- A 3D convolutional neural network to classify subjects as Alzheimer's disease, frontotemporal dementia or healthy controls using brain 18F-FDG PET
- Disentangling temporal and rate codes in primate somatosensory cortex
- Neural critic learning with accelerated value iteration for nonlinear model predictive control
- A robust multi-scale feature extraction framework with dual memory module for multivariate time series anomaly detection
- Adaptive spike threshold dynamics associated with sparse spiking of hilar mossy cells are captured by a simple model
- Exploring Types of Photonic Neural Networks for Imaging and Computing-A Review
- Enhancing neural encoding models for naturalistic perception with a multi-level integration of deep neural networks and cortical networks
- A framework for the definition and interpretation of the use of surrogate endpoints in interventional trials
- Bimodal alteration of cognitive accuracy for spintronic artificial neural networks
- Bimodal alteration of cognitive accuracy for spintronic artificial neural networks
- Proton Conducting Neuromorphic Materials and Devices
- Editorial: Theoretical advances and practical applications of spiking neural networks
- Reporting of surrogate endpoints in randomised controlled trial reports (CONSORT-Surrogate): extension checklist with explanation and elaboration
- Reporting of surrogate endpoints in randomised controlled trial protocols (SPIRIT-Surrogate): extension checklist with explanation and elaboration
- Contrastive Graph Pooling for Explainable Classification of Brain Networks
- Online continual decoding of streaming EEG signal with a balanced and informative memory buffer
- Acute Exercise Improves Large-Scale Brain Network Segregation in Healthy Older Adults
- Motion sensitive network for action recognition in control and decision-making of autonomous systems
- EWT: Efficient Wavelet-Transformer for single image denoising
- Determinantal point process attention over grid cell code supports out of distribution generalization
- Federated learning using model projection for multi-center disease diagnosis with non-IID data
- Neural Correlates of Balance Skill Learning in Young and Older Individuals: A Systematic Review and Meta-analysis
- Deep Learning and Multimodal Artificial Intelligence in Orthopaedic Surgery
- Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain
- Dense Sample Deep Learning
- Dense Sample Deep Learning
- Exploring aperiodic activity in first episode schizophrenia spectrum psychosis: A resting-state EEG analysis
- Exploring aperiodic activity in first episode schizophrenia spectrum psychosis: A resting-state EEG analysis
- Persistent Interruption in Parvalbumin Positive Inhibitory Interneurons: Biophysical and Mathematical Mechanisms
- Biodegradable Oxide Neuromorphic Transistors for Neuromorphic Computing and Anxiety Disorder Emulation
- Training an Ising machine with equilibrium propagation
- Reorganization of the cortical connectome functional gradient in age-related hearing loss
- Sharing massive biomedical data at magnitudes lower bandwidth using implicit neural function
- Triple-0: Zero-shot denoising and dereverberation on an end-to-end frozen anechoic speech separation network
- Astrocyte's self-repairing characteristics improve working memory in spiking neuronal networks
- Dual-channel neural network for instance segmentation of synapse
- Interaction between Functional Connectivity and Neural Excitability in Autism: A Novel Framework for Computational Modeling and Application to Biological Data
- The Classification of Lumbar Spondylolisthesis X-Ray Images Using Convolutional Neural Networks
- Neural waves and computation in a neural net model II: Data-like structures and the dynamics of episodic memory
- Brain functional gradient and structure features in adolescent and adult autism spectrum disorders