Recent changes to feature-requests

Consolidate and document harness exit codes

Andreas Sewe — Thu, 30 Aug 2012 12:00:15 -0000

Currently, there are numerous calls to System.exit with magic numbers strewn about the harness's code. There should be proper constants defined for those in a single place.

patching/modification and incremental building

Sergey Vorobyev — Mon, 27 Sep 2010 12:45:48 -0000

We use daCapo benchmark in open source java race detector develop process.
I fix your ant build scripts and now I can change apache tomcat source and it doesn't override.
I often make minor changes and exec 'ant tomcat'.
Build process take about 10 min.

I think it's good idea and useful for developers allow patching/modification and/or incremental building.

Distribute a policy file with the benchmark suite

Andreas Sewe — Mon, 30 Aug 2010 13:57:30 -0000

Such a policy file is useful when developing a benchmark (bugs <https://sourceforge.net/tracker/?func=detail&atid=861957&aid=3056006&group_id=172498> and <https://sourceforge.net/tracker/?func=detail&atid=861957&aid=3056019&group_id=172498> were found this way); if a benchmark reads files outside the scratch directory, for example, this manifests itself in a permission check.

Attached is a policy file that grants the harness the necessary permissions. It does not yet grant any permissions to a benchmark, however, as it is only meant as a template. Running the benchmark with such a policy is straight-forward:
java -Djava.security.manager -Djava.security.policy=dacapo.policy -Djava.security.debug=failed -jar dacapo-9.12-bach.jar

option to print supported sizes per benchmark

Eric Bodden — Thu, 11 Mar 2010 17:06:53 -0000

Right now, for a shell script, it is not easy to figure out which input sizes are valid for each benchmark. You only find out that the input size is invalid when executing the benchmark and seeing it fail. It would be nice to have an option that returns the list of valid sizes when given a benchmark. Something like...

$java -jar dacapo.jar -sizes fop
small default
$

That would make scripting much easier.

Move much of daytrader.patch into files

Steve Blackburn — Sat, 20 Feb 2010 05:18:24 -0000

daytrader.patch is large and contains a lot of entirely new files. Would be nice if they could be moved out of the patch and into (editable) files. This would make it easier to understand and easier to edit.

clojure and other JVM-targeting languages as workloads?

Steve Blackburn — Sat, 09 Jan 2010 03:37:06 -0000

We should seriously consider adding clojure and jruby (among others) to the suite, just as we currently have jython.

http://clojure.org/

Provide better information upon validation failure

Steve Blackburn — Fri, 04 Dec 2009 04:53:51 -0000

Currently when a benchmark fails validation, the user just gets a message stating that validation failed, and the observed and expected checksums.

Ideally we would have captured the expected output and when validaton fails, we output the diff between expected and observed outputs.

This feature requested by Alexei Svitkine.

Command line option for system.gc() between iterations

Steve Blackburn — Fri, 27 Nov 2009 03:36:17 -0000

Add a command line option that forces system.gc() between iterations. We can have this turned on by default.

Terse summary of what bm is doing

Steve Blackburn — Sat, 07 Nov 2009 01:05:27 -0000

Users often want to know what the difference is between diffefrent "sizes" of each benchmark. This can be established via the .cnf file, but even then is often obtuse.

We should document the affect of size choice in the cnf file as intelligible prose, and allow this to be printed (via the -i command line option probably)

Be explicit about threading model

Steve Blackburn — Tue, 20 Oct 2009 02:54:32 -0000

The adaptive threading model (N per h/w thread) may appear mysterious. We should probably output an explicit message at the start of each execution of each benchmark stating how many threads are running (and in the case of N per h/w thread, state the reason why (ie that there are N h/w threads detected)).