SPMF’s architecture (4) The MemoryLogger

Today, I will continue the series of blog posts to explain the architecture of the SPMF data mining library, and I will talk in particular about a module in SPMF called the MemoryLogger. This module is responsible for recording the memory usage of the different algorithms that are run for statistics purpose.

The MemoryLogger module is located in the package ca.pfv.spmf.tools of SPMF. Here is a picture of the four functions offered by the MemoryLogger:

The SPMF Memory Logger

Descriptions of the functions offered by the MemoryLogger

The function getInstance() must be used to obtain access to the only instance of the MemoryLogger.

Then, after obtaining an instance of the MemoryLogger, we can call the function reset() to reset the recorded memory usage to zero.

Then, if we call the method checkMemory(), the MemoryLogger will check the current memory usage of the Java Virtual Machine. If it is greater than the previously recorded memory usage, it will keep the new value so as to keep the maximum memory usage until now.

Finally, we can call the method getMaxMemory() to obtain the maximum memory usage recorded until now by the memory logger, in Megabytes.

Example of how to use the MemoryLogger

Now let me show you some example code of how to use the Memory Logger tool to compare the memory usage before and after creating some very big integer arrays:

import ca.pfv.spmf.tools.MemoryLogger;

public class MemoryLoggerTest {

	public static void main(String[] args) {
		// Reset the recorded memory usage
		MemoryLogger.getInstance().reset();
		
		// Check the memory usage
		MemoryLogger.getInstance().checkMemory();
		
		// Print the maximum memory usage until now.
		System.out.println("Max memory : " + MemoryLogger.getInstance().getMaxMemory());
		
		int[][] array = new int[99999][9999];
		
		// Check the memory usage
		MemoryLogger.getInstance().checkMemory();
		
		// Print the maximum memory usage until now.
		System.out.println("Max memory : " + MemoryLogger.getInstance().getMaxMemory());
	}

}

The result of running this program is:

maximum memory usage example

This indicates that the memory usage was about 1.8 MB before creating the integer array and about 3848 MB after that.

How the MemoryLogger is used in SPMF?

The Memory Logger is used in SPMF by almost all data mining algorithms to measure their performance. For example, when running an algorithm such as RuleGrowth, it will use the MemoryLogger to record its memory usage and finally display its maximum memory usage when the algorithm terminates as show below:

rulegrowth output

Source code of the MemoryLogger

If you are curious, here is the code of the MemoryLogger. It is very simple:

Conclusion

In this short blog post, I have described a useful tool offered in SPMF called the MemoryLogger. In upcoming blog posts, I will continue explaining other key components of the SPMF software.

==
Philippe Fournier-Viger is a full professor  and the founder of the open-source data mining software SPMF, offering more than 250data mining algorithms. 

This entry was posted in Data Mining, Data science, open-source, Pattern Mining, spmf and tagged , , , , , , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *