Today, I will talk about an interesting topic in academia which is the controversy around ELM (Extreme Learning Machine) and its origins. This has been a hot topic of discussion in the field of machine learning for more than a decade, when some researchers started to question the high similarity of ELM to other models published before such as RBF (Radial Basis Function). There has also been recently some researchers arguing about the similarities between ELM and RVFL (Random Vector Functional Link) and other models.
In this blog post, I will give an overview of this controversy and impact but I will not take any sides. I will just look at it from an outsider’s persective. You can read the arguments from both sides and make your opinion and draw your own conclusions.
Some arguments against ELM
ELM was proposed in 2004. The controversy around the origins of ELM started around 2008 with a letter in IEEE transactions that claimed that it is unecessary to give a new name to a model that existed already with perhaps minor modifications:
- L. P. Wang and C. R. Wan, “Comments on “The Extreme Learning Machine,” in IEEE L. P. Wang and C. R. Wan, “Comments on ‘The extreme learning machine’,” IEEE Trans. Neural Networks, Vol. 19, No. 8, 1494-1495, 2008.
Other researchers have raised this issue. And to understand this perspective, there is an anonymous website that provides a good summary of the issues raised by some researchers against ELM. It is called : ELM Origin (webs.com)
A problem with this website though is that it is anonymous, which means that we cannot be sure who wrote it. However, the website provides annotated ELM papers and claim that several ELM models are similar to papers published many years before. For example, it is said that ELM-Kernel is similar to LS-SVM with zero bias and kernel ridge regression.
I did not read the information in details asthis is outside my main research field so I am personally not sure whether all the claims are reasonable or not.
Some arguments for ELM
There has been researchers that have responsed to these claims to support that there are indeed differences between ELM and previous work. For example:
- G.-B. Huang, “Reply to comments on ‘the extreme learning machine’,” IEEE Trans. Neural Networks, vol. 19, no. 8, pp. 1495-1496, Aug. 2008.
- G.-B. Huang, “What are Extreme Learning Machines? Filling the Gap between Frank Rosenblatt’s Dream and John von Neumann’s Puzzle,” Cognitive Computation, vol. 7, 2015.
However, some researchers argue that these differences are tiny. It was also argued in the defense of ELM that researchers may have simply missed some related work and thus not been aware of the prior work. This might be true… as it has happened in the past that some discoveries were made independently by several researchers.
Yann LeCun’s opinion
One of the fathers of deep learning has also given his opinion on this topic in a Facebook post:
He was clearly not impressed by ELM. However, this is just a Facebook post and it seems that LeCun perhaps did not read all the papers about ELM to have a clear idea about the topic (perhaps?).
Who is right?
As I said previously, I will not take position as this is not my main area. You may make your own mind or write your opinion in the comment section below if you have one.
What is the impact of this controversy?
This controversy has resulted in a kind of war between some researchers working in that area. I have observed that there are researchers against ELM and some that are for ELM that have been quite aggressive towards each other, and there are also many researchers that do not want to take sides but are caught between the two sides.
As I work as associate editor for various journals I have noticed for example, at some point that a reviewer wanted to directly reject a paper just for using the name of ELM. I also noticed some researchers that tried to push their citations against ELM or for ELM. In other cases, I have also seen some reviewer arguing that authors should change their paper because it had shown that ELM was better than some other models and the reviewer could not accept that conclusion, even arguing that this must have been due to experimental errors.
I personally dont really know what to think about this. But as an outsider, it seems to me that today, there is still a kind of war on this topic involving various people, and I think it is a pity for the people who are caught in the middle of that war but do not want to take side.
Conclusion
This is a short blog post to talk about the controversy around ELM. I just report about this topic, as I think it is interesting. As said above, you can read about it and make your own opinion. But personally, I think it is better to not take any side to avoid conflicts.
