Today, I will briefly talk about an upcoming features of SPMF that I am working on. It is file types. Currently, a problem in SPMF is that there are a lot of different input and output formats for the different algorithms and most of them are text files. To make this more manageable, in SPMF 2.67, the input and output format for files will slightly change to add additional metadata lines that are optional and the format will be mainly backward compatible with previous versions of SPMF (except for a few algorithms).
This is best explained with an example. Previously, the Apriori algorithm was taking as input a file in this format where each line is a transaction:
1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5
In SPMF 2.67, the input file format for Apriori will be revised to use this format instead:
@FILETYPE=”Simple transaction database”
@SOURCE=”SPMF SOFTWARE https://philippe-fournier-viger.com/spmf/“
1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5
The difference are two optional lines of metadata that indicate the type of data and the source of the data. This metadata may not seem very useful but it is actually very important to allow SPMF to automatically detect the type of data in a file. For example, this can allow SPMF to recommend suitable algorithms for a given file in the revised user interface of SPMF, which I will describe in a future blog post. For now, I just talk about the file formats.
In addition to the input formats, the output file formats of algorithms will also be adjusted accordingly. For example, previously, the output format of Apriori was like this:
1 #SUP: 3
2 #SUP: 4
3 #SUP: 4
5 #SUP: 4
1 2 #SUP: 2
1 3 #SUP: 3
1 5 #SUP: 2
2 3 #SUP: 3
2 5 #SUP: 4
3 5 #SUP: 3
From version 2.67 of SPMF, the output will be like this:
@FILETYPE=”Frequent itemsets”
@SOURCE=”SPMF SOFTWARE https://philippe-fournier-viger.com/spmf/“
1 #SUP: 3
2 #SUP: 4
3 #SUP: 4
5 #SUP: 4
1 2 #SUP: 2
1 3 #SUP: 3
1 5 #SUP: 2
2 3 #SUP: 3
2 5 #SUP: 4
3 5 #SUP: 3
Again, this will be helpful for further improvements of SPMF. It will simplifies the detection of the content of a file, which opens many other possibilities.
Today, I just brought you a quick overview about this upcoming change. I still have to finish implementing this in SPMF. And actually, it is not a simple change to do as it requires updating over 300 algorithms and also the documentation of those algorithms, and I can only work on this in my free time. I will give more details about the next version 2.67 soon!
—
Philippe Fournier-viger, Ph.D. is a professor of Computer Science and founder of the SPMF data mining software.
