RATE is a general platform for biometrics algorithm evaluation and data exploration. It aims at providing a convenient tool for doing biometric research, it solves various problems in conducting biometric experiment - data management and annotation, customizable benchmark and high computation cost in evaluation.
There are 4 main concepts in RATE, they are described in the following sections. Here is the guide to use RATE.
View refers to collection of samples (e.g. a group of fingerprint images). View is useful when researchers want to group samples with specific research propose. For example, finger vein verificaion researchers may want to know the general performance on different fingers, they could then create 3 views for index finger, middle finger, and ring finger (finger vein verification just does not like thumb or little finger).
Basically, you upload an algorithm, and RATE runs it for you. There should be a place that you describe the detail of which sample is going to match with which another one - this is called benchmark in RATE. Benchmark is created upon view, all samples referred in a benchmark should be in one specific view. There are many ways to automatically generate a benchmark:
Algorithm is two Win32 excutables in RATE: enroll.exe and match.exe, for the enrollment step and the matching step. The two executables should read input from the command line arguments and output results to the standard output or a designated file with the right exit value. They must obey the following rules
|Executable||Decription||Command line args||Standard output||Exit value||Time Limit||Memory limit|
|Enroll.exe||Create template||Paths of the input images and output template file||None||0: success; Else: fail||30s||2048MB|
|Match.exe||Match two templates||Paths of two templates for matching||Similarity degree is indicated by a float number between [0-1.0], with 0 meaning not similar at all, and 1 meaning exact the same||0: success; Else: fail||10s||2048MB|
Keep in mind that the right exit value is very important, and you may fail the evaluation if you don’t care about it.
The executables will be used in a command line like follows:
> Enroll.exe \path\to\read\image.bmp \path\to\write\template.t > Match.exe \path\to\template1.t \path\to\template2.t
Task is the result of running an algorithm on a benchmark. RATE's distributed system will run the task for you on a given benchmark, when done, it will run an automatic analysis process.