Recent changes to feature-requests

Consolidate and document harness exit codes

2012-08-30T12:00:15Z

Currently, there are numerous calls to System.exit with magic numbers strewn about the harness's code. There should be proper constants defined for those in a single place.

patching/modification and incremental building

2010-09-27T12:45:48Z

We use daCapo benchmark in open source java race detector develop process.
I fix your ant build scripts and now I can change apache tomcat source and it doesn't override.
I often make minor changes and exec 'ant tomcat'.
Build process take about 10 min.

I think it's good idea and useful for developers allow patching/modification and/or incremental building.

Distribute a policy file with the benchmark suite

2010-08-30T13:57:30Z

Such a policy file is useful when developing a benchmark (bugs <https://sourceforge.net/tracker/?func=detail&atid=861957&aid=3056006&group_id=172498> and <https://sourceforge.net/tracker/?func=detail&atid=861957&aid=3056019&group_id=172498> were found this way); if a benchmark reads files outside the scratch directory, for example, this manifests itself in a permission check.

Attached is a policy file that grants the harness the necessary permissions. It does not yet grant any permissions to a benchmark, however, as it is only meant as a template. Running the benchmark with such a policy is straight-forward:
java -Djava.security.manager -Djava.security.policy=dacapo.policy -Djava.security.debug=failed -jar dacapo-9.12-bach.jar

option to print supported sizes per benchmark

2010-03-11T17:06:53Z

Right now, for a shell script, it is not easy to figure out which input sizes are valid for each benchmark. You only find out that the input size is invalid when executing the benchmark and seeing it fail. It would be nice to have an option that returns the list of valid sizes when given a benchmark. Something like...

$java -jar dacapo.jar -sizes fop
small default
$

That would make scripting much easier.

Move much of daytrader.patch into files

2010-02-20T05:18:24Z

daytrader.patch is large and contains a lot of entirely new files. Would be nice if they could be moved out of the patch and into (editable) files. This would make it easier to understand and easier to edit.

clojure and other JVM-targeting languages as workloads?

2010-01-09T03:37:06Z

We should seriously consider adding clojure and jruby (among others) to the suite, just as we currently have jython.

http://clojure.org/

Provide better information upon validation failure

2009-12-04T04:53:51Z

Currently when a benchmark fails validation, the user just gets a message stating that validation failed, and the observed and expected checksums.

Ideally we would have captured the expected output and when validaton fails, we output the diff between expected and observed outputs.

This feature requested by Alexei Svitkine.

Command line option for system.gc() between iterations

2009-11-27T03:36:17Z

Add a command line option that forces system.gc() between iterations. We can have this turned on by default.

Terse summary of what bm is doing

2009-11-07T01:05:27Z

Users often want to know what the difference is between diffefrent "sizes" of each benchmark. This can be established via the .cnf file, but even then is often obtuse.

We should document the affect of size choice in the cnf file as intelligible prose, and allow this to be printed (via the -i command line option probably)

Be explicit about threading model

2009-10-20T02:54:32Z

The adaptive threading model (N per h/w thread) may appear mysterious. We should probably output an explicit message at the start of each execution of each benchmark stating how many threads are running (and in the case of N per h/w thread, state the reason why (ie that there are N h/w threads detected)).