User Manual

User Manual for OCELOT

Installation

OCELOT requires:

• Java Development Kit (JDK) >= 7.0

• glib 2.0

• Cygwin (for Windows)

OCELOT was tested on both Linux e Mac OSX and it is fully supported on those. It is possible to run OCELOT on Windows, but you may need some specific setup.

Dependencies

If you use a Linux distribution with Gnome DE, you are ready to go. Otherwise, you may need to install glib2. For example:

sudo apt-get install libglib2.0-0 libglib2.0-dev

On MacOS, you need to install glib from brew:

brew install glib

Configuration

To generate test data, it is necessary to configure OCELOT through a properties file (config.properties). This is a precise description of the parameters that can be set:

Build

Parameters regarding the instrumentation of the target C function.

• config.javaHome: Path to the system JDK.

/usr/lib/jvm/java-8-openjdk

• config.systemInclude: Comma-separated list of the system's include paths.

/usr/include

• config.glib2Paths: Comma-separated list of the system's glib2 paths.

/usr/include/glib-2.0/,/usr/lib/glib-2.0/include/

• config.javaPaths: Comma-separated list of the path to the JDK JNI libraries.

/usr/lib/jvm/java-8-openjdk/include

• config.cflags: Additional gcc compile flags

-shared -fpic

• config.options: Additional gcc compile options

-lrt

• config.make: System's make command

make

• test.basedir: Base direcory of the project. Every other path in the config file is relative to this directory. Note: this parameter has to end with a slash (“/”).

/home/user/Project/src/

• test.filename: Name of the file containing the target function.

toTest.c

• test.includes: Comma-separated list of header files needed by the target function

../include/,../include2/,/usr/lib/nonstandard/include/

• test.function: Name of the target function (no formal parameters)

test

• test.parameters.ranges: Space-separated list of ranges of values for the inputs of the target function. The number of ranges has to be equa l to the number of formal primitive parameters. When a parameter is of type structure, it is necessary to specify the range of each primitive type in the structure. The format of a single range is of format “min:max”

-100:100 -100:100 -10000:10 0:10000

• test.arrays.size: Indicates the maximum size of the arrays generated by OCELOT. It is suggested to have small values to avoid huge search spaces.

20

• test.links: List of static libraries required by the test function (e.g., containing functions called by such a function).

./test1.so,./test2.so,./test3.so

Experiments

Parameters regarding the execution of experiments.

• experiment.output.folder: Output directory for OCELOT logs

• experiment.results.folder: Output directory for results

• experiment.results.print: Print results on the standard output. It has to be true o false.

• experiment.runs: Number of runs of the experiment

Algorithms

Parameters regarding the search algorithms.

• suite.generator.algorithm: Algorithm to use to generate test data. For example, AVM or *GeneticAlgorithm*. Note that some strategies require their own algorithm. For those, this parameter is ignored. See `suite.generator` for details.

AVM

• population.size: Size of the population (for Genetic Algorithms)

100

• crossover.probability: Probability of using the crossover operator (for Genetic Algorithms)

0.8

• mutation.probability: Probability of using the mutation operator (for Genetic Algorithms).

0.2

• operators.mutator.metamutator: Switch for a mutation strategy that randomly mutates the values using constants of the target program (for Genetic Algorithms). It has to be true or false.

• avm.epsilon: Minimum variation considered (for AVM).

0.00000000001

• avm.delta: Speed of the increment/decrement in AVM. The larger, the higher the exploration of the search space.

1

Strategies

Parameters regarding the search strategies (generators)

• evaluations.max: Maximum number of evaluations of the target function.

100,000

• suite.generator: Strategy chosen to generate a test suite. OCELOT implements some state-of-the-art generators out of the box. See below for a complete list of available generators.

DynamicMcCabe

• suite.generator.cascade: Comma-separated list of strategies to be run in cascade (if the strategy is Cascade). All the strategies will share the same configuration.

Random,Minimizer

• suite.generator.serendipitous: Accept or discard serendipitous test data. It has to be true or false.

• suite.generator.budgetmanager: Indicates the budget management strategy to use for the strategies (except for MOSA). At the moment, it is possible to use only Basic.

Basic

• suite.minimizer: Indicates the test-suite minimization algorithm to be used. At the moment, only the additional greedy algorithm is available (AdditionalGreedy).

AdditionalGreedy

• suite.coverage: Target coverage. Once it is reached, the test generation process stops.

0.95

Debug

Parameters for debugging. We recommend not to use them. Can either be true or false.

• test.debug: Prints additional information

• test.ui: Prototype parameter that allows to visualize the Control Flow Graph of the target function

Strategies (generators)

The generators available in OCELOT are the following:

• MOSA – A many-objective algorithm. Uses its own search algorithm.

• DynamicMcCabe – LIPS strategy. Accepts both GeneticAlgorithm and AVM as search algorithm.

• McCabe – Statically generates linearly independent paths and tries to cover them. Slower than other strategies. Accepts both GeneticAlgorithm and AVM as search algorithm.

• Cascade – Allows to combine many strategies (see suite.generator.cascade).

• MemoryEdges – Randomly choses branches to cover and ignores the branches already covered. Accepts both GeneticAlgorithm and AVM as search algorithm.

• Minimizer – It minimizes the test suite (if combined in cascade)

• Random - Random test data generation strategy

Execution

To execute OCELOT from command line using, write:

java -Djava.library.path=. -jar Ocelot.jar

It is possible to specify some parameters after the name of the jar:

• type=value: Type of execution. Chose one of the following:

  • simple (looks for test data and prints the obtained coverage);

  • experiment (runs an experiment);

  • write (writes skeletons of test suites).

  • The default type is write.

java -Djava.library.path=. -jar Ocelot.jar type=experiment

• --build|-b: Forces the compilation of the target function. It is suggested not to use this parameter, OCELOT automatically detects when it has to re-compile the function.

java-Djava.library.path=. -jar Ocelot.jar -b

• --no-build|-B: Forces OCELOT not to compile the target function. Do not use this unless you are fully aware of what you are doing.

java-Djava.library.path=. -jar Ocelot.jar -B

• --version|-v: Prints the OCELOT version

java -Djava.library.path=. -jar Ocelot.jar -v

• config=value: Uses a specific configuration file

java-Djava.library.path=. -jar Ocelot.jar config=y.cfg

• expgen=value: Forces the use of specific strategies in the experiment mode. The generators have to be comma-separated.

java -Djava.library.path=. -jar Ocelot.jar expgen=McCabe,DynamicMcCabe

• parameter=value: Overwrites a configuration parameter. It is not effective if you use the config option after this (it will overwrite the parameter).

java -Djava.library.path=. -jar Ocelot.jar avm.delta=2

Test suite execution

OCELOT uses external frameworks for unit test definition. At the moment, OCELOT supports only check.

Oracle definition

After running OCELOT in write mode, OCELOT will output the generated skeleton of test suite in a file named "\_Test\_*FNC*.c", where "FNC" is replaced by the name of the target function.

In such a file, you can find occurrences of the string "/* REPLACE THE ASSERTION BELOW */” to find the points in which it is necessary to add an assertion. The default assertion used by OCELOT is the following:

ck_assert_str_eq("OK", "OK");

Look at the check home page to define approppriate assertions.