fMRI-based Decoding of Visual Information from Human Brain Activity: A Brief Review

  • Share:

One of the most significant challenges in the neuroscience community is to understand how the human brain works. This paper, by Prof. Zhang Daoqiang of Nanjing University of Aeronautics and Astronautics, mainly provides a comprehensive and up-to-date review of machine learning methods for analyzing neural activities with the following three aspects, i.e., brain image functional alignment, brain activity pattern analysis, and visual stimuli reconstruction. In addition, online resources and open research problems on brain pattern analysis are also provided for the convenience of future research.

 

Download full text:

fMRI-based Decoding of Visual Information from Human Brain Activity: A Brief Review

Shuo HuangWei ShaoMei-Ling WangDao-Qiang Zhang

http://www.ijac.net/en/article/doi/10.1007/s11633-020-1263-y

https://link.springer.com/article/10.1007/s11633-020-1263-y

 

 

One of the most significant challenges in the fields of neuroscience and machine learning is comprehending how the human brain works. As the provenance of human memory, emotion and thoughts, a better comprehension of the brain will expedite the rapid development of society, including science, medicine, education, etc.

 

In order to measure neural activities, different modalities of measurement can be utilized, including event-related optical signals (EROS), positron emission tomography (PET), single-photon emission computed tomography (SPECT), near-infrared spectroscopy (NIRS), magnetoencephalography (MEG), electrocorticography (ECoG), electroencephalography (EEG), and functional magnetic resonance imaging (fMRI). Among all of the above imaging biomarkers, fMRI is one non-invasive technique for probing the neurobiological substrates of various cognitive functions that can provide indirect estimation of brain activity and measure the metabolic changes in blood flow. Another advantage of fMRI is that it can provide unprecedented spatiotemporal resolution without known side effects, which intuitively can provide more accurate information for the analysis of neural activities. 



Based on the fMRI images, many machine learning models are applied to analyse the visual and subjective contents of human brains. Generally, the machine learning-based methods aim to build a mathematical model based on the fMRI sample data, namely the training data, in order to make predictions or decisions without being explicitly programmed to perform the neural activity prediction task on the testing set. 

 

Although much progress has been achieved, given the data sets for the analysis of brain activities, major computational and statistical challenges have arisen to realize the full unprecedented scale and complexity of the valuable fMRI data. Overcoming these challenges has become a major and active research topic in the fields of statistical and machine learning.

 

Here, this paper summarizes and lists the main challenges for brain pattern analysis as follows: First of all, a key component of fMRI research will be the use of multi-subject datasets. However, both anatomical structure and functional topography (brain activity patterns) vary across subjects, and thus the authentic functional and anatomical alignments among different subjects′ neural activities should be addressed before the development of the classification models.

 

Secondly, the dimensionality of fMRI datasets is always high with redundant noise. For some specific brain research experiments, such as visual or auditory stimulation, only a part of the brain area is activated in these tasks. Selecting key brain areas is a prerequisite for accurate brain research.

 

Last but not the least, although researchers have successfully improved the classification performance for identifying brain activity patterns, the reconstruction of visual stimuli via brain images is still a challenging task. Compared with the classification tasks, reconstruction of visual images can provide more detailed information for understanding human minds.

 

In recent years, some reviews reviewed the mechanisms of brain encoding and decoding as well as common and classic methods. These reviews not only summarized the up-to-date methods, but also presented the challenges in the field of brain decoding and neuroscience. In view of the above challenges, the majority of this review will be devoted to the discussion of the machine learning algorithms for solving the following four types of problems in the field of brain decoding.

 

 

Firstly, in Section 2, the paper will examine the problem of functional alignment for fMRI analysis across subject, which is a pre-processing step for the brain decoding analysis that takes into account variability between subjects. Since most of the research reviewed here belongs to this category, this paper will review a few fundamental brain alignment strategies in Section 2 including linear functional alignment, non-linear functional alignment, etc. Secondly, in Section 3, this paper will explore the problems of multivariate pattern classification and representation similarity analysis that predict the neural patterns with distinctive stimuli, as well as evaluate the similarities (or distances) between different cognitive tasks. Thirdly, in Section 4, this paper will review the methods for brain image reconstruction that generate the stimuli image via corresponding fMRI signals. Finally, online resources and open research problems on brain pattern analysis are also provided in Section 5.


easy fMRI (open source):

https://easyfmri.learningbymachine.com/ 

 

 

Download full text:

fMRI-based Decoding of Visual Information from Human Brain Activity: A Brief Review

Shuo HuangWei ShaoMei-Ling WangDao-Qiang Zhang

http://www.ijac.net/en/article/doi/10.1007/s11633-020-1263-y

https://link.springer.com/article/10.1007/s11633-020-1263-y

 

◇◇◇◇◇◇

For more up-to-date information:

1) WeChat: IJAC

2) Twitter:IJAC_Journal

3) Facebook:International Journal of Automation and Computing

4) Linkedin: Int.J. of Automation and Computing

5) Sina Weibo:IJAC-国际自动化与计算杂志

  • Share: