Frequently Asked Questions

What is the Birmingham Environment for Software Testing?

The Birmingham Environment for Software Testing (BEST) is software testing framework designed specifically for science. It runs numerical models many times and in parallel to ensure that the predictions are correct. Each test has two components the testfile and the model, often these are written in different languages and BEST is designed to cope with this. BEST is designed to be very versatile, with support for grouping types of errors based on return codes and catching relative errors from output files.

How is it built?

BEST is built on top of Docker and Git to provide each test with a traceable environment and code base. The UI is written in python-flask and Javascript. BEST can be deployed on any server and runs in its own python virtual environment.

Why is it sometimes called BrumSoftTest?

BrumSoftTest is an old name for BEST, before the first alpha release. It is still used for variable names and configuration files as these need to be globally unique, but not too long.

Why are uploaded files missing?

Sometimes uploaded files are missing. Most probably they were not generated in the first place. Check the logs in stdout and stderr, to see if any file was written.

If is old, it may have been deleted. The database entries are kept indefinitely, but uploaded files are purged from time to time to maintain disk space. If this is the case re-run your test and the files will be available.

If the log suggests the files were uploaded and you can’t see a link to them, email the sysadmin there may be a bug in BEST.

Why must I be logged in to see the datastore?

Data in the datastore matches the visibility of the associated project. If you are trying to view data based on a commit SHA1, then you must always be logged in.

Why must I be logged in to see complete documentation?

Complete documentation includes server administration information and source code until such a time as we are satisfied with the security in place for BrumSoftTest we are restricting access to users only.

In the mean time, non logged in users can see the user guide FAQ and support information only.