An important but often overlooked part of any software project is the management of the build process, in which a group of source code files are converted into libraries and applications for the end user. A programming system usually provides a set of tools for this purpose, consisting of compilers, specialized generators, linkers, and archivers.
Coordinating these tools between all computers and developers is quite difficult and time consuming. The tools and how they are used vary greatly from platform to platform, so managing the build process for Windows, Unix, and Mac OS at the same time is quite difficult.
Before the advent of the CMake tool, Kitware developers on Unix used the autoconf utility, and on Windows they used a specialized (taking into account the features of the Visualization Toolkit) nmake makefile generator. Makefiles were compiled by hand, including generated lists of source files. This process was error-prone and had a number of limitations, so few of the developers knew how to make changes to the assembly.
The tool receives simple script files in the CMake language as input. In accordance with these scenarios, an internal analysis of the system is performed and a description of the process of building executable programs, libraries, and other necessary target files is compiled. Unlike many other cross-platform build tools, CMake is designed to be used in conjunction with a native build environment by generating build control files for “native” tools familiar to developers.
This is important for small companies, because implementing unified tools requires training and can cause resentment among developers who are forced to use a build environment imposed by corporate standards. Large organizations have the ability to have separate teams dedicated to the build process. For its part, by automating much of the work involved, CMake enables even small teams to successfully develop portable software. With a small script, you can describe how to build an executable program on multiple platforms.
CMake has been actively developed since 1999 and has grown into a powerful general purpose tool with a growing user base. It was recently chosen by the K Desktop Environment community to replace autoconf and Scons (KDE is one of the largest Open Source projects). CMake is now a mature, full-featured tool that gives you full control over the build process and can replace the separate build team that many large organizations have. The simple syntax of the CMake language allows any developer to easily create the executable programs and libraries necessary for the project.
Testing
- Test types.
Testing of the software complex can take place in various forms. At the lowest level are the so-called “smoke” tests (a rough simple-run sanity check), which simply check that the software compiles without errors. As simple as they may seem, with a wide variety of platforms and possible configurations, smoke tests reveal more problems than any other type of test. Another form of application of such tests is to check the failure-free operation.
It can come in handy when a developer doesn’t want to take the time to create complex tests, but wants to run some of the simple ones. These simple tests are often small programs that you can run to check not only that the build succeeded, but that all required shared libraries can be loaded (in projects that use them) and that at least some of the code can run smoothly.
Basic tests are followed by more meaningful tests such as regression testing and black-box and white-box testing. When a regression test fails, a quick look at recent code changes can usually help you understand why. Unfortunately, creating regression tests usually requires more effort than other types of tests.
The final type of testing is compliance testing. While other types of tests focus on the correct functioning of the code, conformance testing is designed to determine whether the code meets the programming standards adopted by the project. For example, it allows you to make sure that some key method is implemented in all classes, or that all functions have a common prefix. For this type of testing, the number of options is endless, and it can be done in a variety of ways.
- Testing approach.
We have developed a simplified procedure for compiling a set of tests and a simple mechanism for executing these tests and viewing their results. In small or short projects, testing is often limited to assembly verification and smoke tests. On larger and longer projects, we put more effort into testing and create custom regression tests that cover key software features.
We use a range of tools to conduct testing. The first is CTest, which Kitware ships with CMake. The CTest program reads test definition files, usually generated by CMake, and allows the user to run some or all of the tests prepared for the project. A typical workflow looks like this: changes are made to the source code, a build is made, CTest is run to verify that the prepared tests pass, after which the changes are submitted to the change control system.
In projects with a large number of tests that can take several hours to complete, we usually limit them to a subset of them, selected by pattern matching, or use a large enough step to skip the tests (say, run every tenth of them). It’s important to have some baseline of testing before committing code to a shared repository that doesn’t cause developers to wait more than a few minutes. More comprehensive testing is carried out using automated tools.
In automated testing, CTest is run on the client machine, but the results are sent to the Dart database, an open web-based test reporting tool developed at GE Global Research. CTest is usually run at night under the control of the cron utility or task scheduler and does the whole cycle of extracting files, building and testing. Typically, one client tests multiple projects or branches overnight and submits entries to Dart, which in turn runs on the server and receives results from many clients.
One server can serve many software projects at the same time and allows you to quickly identify build and test problems on multiple test platforms. On fig. Figure 1 shows the Dart dashboard for CMake. Each row summarizes the test results for an individual client. Hyperlinks allow developers to drill down to see messages specific to any build, including warnings or errors, and the output of any tests.
Typically, clients (usually working at night) submit coordinated data to the dashboard based on the same version of the source code, so that client results can be compared with each other. Dart’s always-on readiness allows the largest projects to continuously display build and test status on some clients as the source code changes throughout the day. Sometimes it helps to find and fix cross-platform issues on the day they occur.
In addition to build and test results, Dart ingests data and reports code test coverage and dynamic analysis results. The code test coverage indicator allows you to quickly assess the amount of code that has passed the testing process. In a Linux testing client, it can be easily configured by changing the compilation flags for one of the nightly runs (for example, using the gcov utility). CTest will automatically detect coverage files, parse them, and send them to Dart, where developers can view them in detail.
For dynamic analysis, we usually use the free Linux toolkit valgrind. It’s also highly configurable, usually by changing the compilation flags slightly and telling CTest the location of valgrind. Dynamic analysis of test reports allows you to identify errors such as writing outside the array, reading data from uninitialized memory, etc. Also useful dynamic analysis tools are Purify and Bounds Checker.