{"id":4394,"date":"2019-01-07T17:10:47","date_gmt":"2019-01-07T15:10:47","guid":{"rendered":"https:\/\/engel-wolf.com\/?p=4394"},"modified":"2019-10-27T17:17:35","modified_gmt":"2019-10-27T15:17:35","slug":"rtest-pretty-testing-of-r-packages","status":"publish","type":"post","link":"https:\/\/engel-wolf.com\/?p=4394","title":{"rendered":"RTest: pretty testing of R packages"},"content":{"rendered":"\n<h3 class=\"wp-block-heading\" id=\"bc23\">The specflow and cucumber.io for R. Enabling non-coders to interpret test reports for R-packages, moreover allowing non-coders to create test cases. A step towards simple r package validation.<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"02c2\">Table of&nbsp;contents<\/h4>\n\n\n\n<ul><li><a href=\"#5cfa\">Why RTest?<\/a><\/li><li><a href=\"#7066\">What\u2019s special about RTest?<\/a><\/li><li><a href=\"#618a\">An example of a test implementation with RTest<\/a>.<\/li><li><a href=\"#85b2\">Further Reading<\/a><\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"5cfa\">Why RTest?<\/h3>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"alignleft\"><img loading=\"lazy\" decoding=\"async\" width=\"137\" height=\"157\" src=\"https:\/\/engel-wolf.com\/wp-content\/uploads\/Picture1.png\" alt=\"\" class=\"wp-image-4396\"\/><\/figure><\/div>\n\n\n\n<p>Testing in R seems simple. Start by using\u00a0<code>usethis::test_name(\"name\")<\/code>\u00a0and off you go by coding your tests in\u00a0<a rel=\"noreferrer noopener\" href=\"http:\/\/testthat.r-lib.org\/\" target=\"_blank\"><em>testthat<\/em><\/a>with functions like\u00a0<code>expect_equal<\/code>. You can find a lot of tutorials online, there is even a whole book on \u201c<a rel=\"noreferrer noopener\" href=\"https:\/\/www.amazon.de\/gp\/product\/1498763650\/ref=as_li_tl?ie=UTF8&amp;camp=1638&amp;creative=6742&amp;creativeASIN=1498763650&amp;linkCode=as2&amp;tag=zappingseb-21&amp;linkId=ca15df0c474acdcf98c8750db741c4e3%22%3ETesting%20R%20Code%20%28Chapman%20&amp;%20Hall\/Crc%20the%20R%29%3C\/a%3E%3Cimg%20src=%22\/\/ir-de.amazon-adsystem.com\/e\/ir?t=zappingseb-21&amp;l=am2&amp;o=3&amp;a=1498763650\" target=\"_blank\">Testing R Code<\/a>\u201d. Sadly, this is not the way I can go. As I mentioned a\u00a0<a href=\"https:\/\/engel-wolf.com\/?p=4322\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\" (\u00f6ffnet in neuem Tab)\">few times<\/a>, I work in a strongly\u00a0<strong>regulated\u00a0<\/strong>environment. Inside such environments your tests are not only checked by coders, but also by people who cannot code. Some of your tests will even be written by people who cannot code. Something like\u00a0<a rel=\"noreferrer noopener\" href=\"https:\/\/specflow.org\/\" target=\"_blank\">specflow\u00a0<\/a>or\u00a0<a rel=\"noreferrer noopener\" href=\"http:\/\/docs.cucumber.io\/guides\/overview\/\" target=\"_blank\">cucumber\u00a0<\/a>would really help them to write such tests. But those do not exist in R. Additionally, those people cannot read command line test reports. You can train them to do it, but we decided it is easier to provide us and them with a pretty environment for testing, called\u00a0<strong>RTest<\/strong>.<\/p>\n\n\n\n<p>If you want to know more about the reasons for developing such an environment, you can read the article:\u00a0<a href=\"https:\/\/engel-wolf.com\/?p=4327\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\" (\u00f6ffnet in neuem Tab)\">Why do we need human readable test for a programming language.<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"7066\">What\u2019s special about&nbsp;RTest?<\/h3>\n\n\n\n<p>To explain which features we put into&nbsp;<a href=\"https:\/\/github.com\/zappingseb\/RTest\" rel=\"noreferrer noopener\" target=\"_blank\"><em>RTest<\/em><\/a>I will start describing of a basic testing workflow.<\/p>\n\n\n\n<p>1 Testing code starts with writing code. Your R-package will contain functions, classes and methods. These shall be tested.<\/p>\n\n\n\n<p>2 Writing the tests now mostly includes calls like this:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: r; title: ; notranslate\" title=\"\">\nmy_function(x,y){sums_up(x,y) return(z)}\nx=3\ny=4\nz=7\nstopifnot(my_function(x,y)==z)\n<\/pre><\/div>\n\n\n<p>It is easy to see, that your test will break if your function&nbsp;<code>my_function<\/code>&nbsp;cannot sum up two values. You will create a bunch of such tests and store them in a separate folder of your package, normally called&nbsp;<code>tests<\/code>&nbsp;.<\/p>\n\n\n\n<p>3 Afterwards you can run all such tests. You can include a script in the&nbsp;<code>tests<\/code>&nbsp;folder or use&nbsp;<em>testthat&nbsp;<\/em>and run<code>testthat::test_dir()<\/code>&nbsp;.<\/p>\n\n\n\n<p>4 If one of your test fails the script will stop and tell you which test failed in the console. This describes the 4 steps shown in the figure below.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/cdn-images-1.medium.com\/max\/1200\/1*KfNxXhuJ_pSyDMtUe4u48w.jpeg\" alt=\"\"\/><\/figure>\n\n\n\n<p>What\u2019s now special about RTest are two major steps.<\/p>\n\n\n\n<ol><li>The definition of the tests<\/li><li>The&nbsp;<a href=\"#b317\">reporting&nbsp;<\/a>of the test execution<\/li><\/ol>\n\n\n\n<p>For the&nbsp;<strong>definition of the tests&nbsp;<\/strong>we decided for XML. Why XML? XML is not just easier to read then pure R-Code, it comes with a feature, that is called XSD; \u201cXML schema definition\u201d. Each XML test case we create can immediately be checked against a schema designed by the developer. It can also be checked against our very own&nbsp;<code>Rtest.xsd<\/code>. This means the tester can double check the created test cases before even executing them. This saves us a lot of time and gives a fixed structure to all test cases.<\/p>\n\n\n\n<p>The&nbsp;<strong>reporting&nbsp;<\/strong>was implemented in HTML. This is due to the many features HTML comes with for reporting. It allows coloring of test results, linking to test cases and including images. The main difference for the reporting in HTML between&nbsp;<em>RTest<\/em>&nbsp;and&nbsp;<em>testthat<\/em>&nbsp;is that RTest reports every test that shall be executed, not only the failed ones. The test report will also include the value created by the function call and the one given as a reference. The reader can see if the comparison really went right. By this the test report contains way more information than the&nbsp;<em>testthat<\/em>&nbsp;console log.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"618a\">An example of a test implementation with&nbsp;RTest<\/h3>\n\n\n\n<p>Please note the whole example is stored in a<a href=\"https:\/\/gist.github.com\/zappingseb\/0f5dabe94c7d284bc543469c50a4213c\" rel=\"noreferrer noopener\" target=\"_blank\">&nbsp;github gist<\/a>. Please star the gist if you like this example.<\/p>\n\n\n\n<ol><li>Given a\u00a0<strong>function\u00a0<\/strong>that sums up two columns:<\/li><\/ol>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: r; title: ; notranslate\" title=\"\">\nmy_function &lt;- function(\n  data = data.frame(x = c(1,2), y = c(1,2))){\n    stopifnot(dim(data)&#91;2] == 2)\n    data&#91;, &quot;sum&quot;] &lt;- apply(data, 1, \n     function(x){sum(x)})\n    return(data)\n}\u00a0\n\n<\/pre><\/div>\n\n\n<hr class=\"wp-block-separator\"\/>\n\n\n\n<p>2. We want to have one successful and one non successful test. Both will have three parts in the\u00a0<strong>XML file<\/strong>:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: xml; title: ; notranslate\" title=\"\">\n&lt;params&gt;&lt;reference&gt;&lt;testspec&gt;\n<\/pre><\/div>\n\n\n<p><code>params<\/code>&nbsp;accounts for input parameters<\/p>\n\n\n\n<p><code>reference<\/code>&nbsp;for the output data.frame<\/p>\n\n\n\n<p><code>testspec<\/code>&nbsp;for whether the test shall run silently and what the tolerance is<\/p>\n\n\n\n<p>For the successful test our test would look like this:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: xml; title: ; notranslate\" title=\"\">\n\n&lt;my_function test-desc=&quot;Test data.frame&quot;&gt;\n  &lt;params&gt;\n    &lt;RTestData_input_data param=&quot;data&quot; name=&quot;test01&quot; \/&gt;\n  &lt;\/params&gt;\n  &lt;reference&gt;\n    &lt;col-defs&gt;\n      &lt;coldef name=&quot;x&quot; type=&quot;numeric&quot; \/&gt;&lt;coldef name=&quot;y&quot; type=&quot;numeric&quot; \/&gt;&lt;coldef name=&quot;sum&quot; type=&quot;numeric&quot; \/&gt;\n    &lt;\/col-defs&gt;\n    &lt;row&gt;\n      &lt;cell&gt;1&lt;\/cell&gt;&lt;cell&gt;2&lt;\/cell&gt;&lt;cell&gt;3&lt;\/cell&gt;\n    &lt;\/row&gt;\n    &lt;row&gt;\n      &lt;cell&gt;1&lt;\/cell&gt;&lt;cell&gt;2&lt;\/cell&gt;&lt;cell&gt;3&lt;\/cell&gt;\n    &lt;\/row&gt;\n  &lt;\/reference&gt;\n  &lt;testspec&gt;\n    &lt;execution execution-type=&quot;silent&quot; \/&gt;\n    &lt;return-value compare-type=&quot;equal&quot; diff-type=&quot;absolute&quot;\n      tolerance=&quot;0.001&quot; \/&gt;\n  &lt;\/testspec&gt;\n&lt;\/my_function&gt;\n<\/pre><\/div>\n\n\n<p><a href=\"https:\/\/gist.github.com\/zappingseb\/9b801907d18cf7c10c85d971bb63b58e\/raw\/48db5b0d00f5f5d0678d9db4bce00056f7cd3124\/RTest_part.xml\">view raw<\/a><a href=\"https:\/\/gist.github.com\/zappingseb\/9b801907d18cf7c10c85d971bb63b58e#file-rtest_part-xml\">RTest_part.xml<\/a>&nbsp;hosted with \u2764 by&nbsp;<a href=\"https:\/\/github.com\">GitHub<\/a><\/p>\n\n\n\n<p>You can immediately see one special feature of RTest. It allows to use data sets for multiple tests, we store those data sets in the&nbsp;<code>input-data<\/code>&nbsp;tag.This saves space in the file. The dataset&nbsp;<code>test01<\/code>&nbsp;will be used here. Moreover a test description can be given for each test. For each data.frame stored in XML the types of the columns can be given in&nbsp;<code>col-defs<\/code>&nbsp;. Here those are all numeric.<\/p>\n\n\n\n<p>The<code>input-data<\/code>is now given here:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: xml; title: ; notranslate\" title=\"\">\n&lt;input-data&gt;\n&lt;data.frame name=&quot;test01&quot;&gt;\n  &lt;col-defs&gt;\n    &lt;coldef name=&quot;x&quot; type=&quot;numeric&quot; \/&gt;&lt;coldef name=&quot;y&quot; type=&quot;numeric&quot; \/&gt;\n  &lt;\/col-defs&gt;\n  &lt;row&gt;\n\t&lt;cell&gt;1&lt;\/cell&gt;&lt;cell&gt;2&lt;\/cell&gt;\n  &lt;\/row&gt;\n  &lt;row&gt;\n    &lt;cell&gt;1&lt;\/cell&gt;&lt;cell&gt;2&lt;\/cell&gt;\n  &lt;\/row&gt;\n&lt;\/data.frame&gt;\n&lt;\/input-data&gt;\n<\/pre><\/div>\n\n\n<p><a href=\"https:\/\/gist.github.com\/zappingseb\/4ce6ee53c93001fc4a58066445b345fc\/raw\/34f7f2be457fc1f5969426ade1b63539b122e55f\/RTest_input_data.xml\">view raw<\/a><a href=\"https:\/\/gist.github.com\/zappingseb\/4ce6ee53c93001fc4a58066445b345fc#file-rtest_input_data-xml\">RTest_input_data.xml<\/a>&nbsp;hosted with \u2764 by&nbsp;<a href=\"https:\/\/github.com\">GitHub<\/a><\/p>\n\n\n\n<p>It\u2019s a data frame with the&nbsp;<em>x<\/em>&nbsp;column just carrying 1 and the&nbsp;<em>y<\/em>&nbsp;column just carrying 2. The test shall create a data.frame with the sum column being 3 in each row.<\/p>\n\n\n\n<p>We can easily let the test fail by changing the&nbsp;<code>reference<\/code>&nbsp;tag and instead of having just 3 in the&nbsp;<code>sum<\/code>&nbsp;column we can add a 3.5 to let the test fail. The whole test case can be found inside the&nbsp;<a href=\"https:\/\/gist.github.com\/zappingseb\/0f5dabe94c7d284bc543469c50a4213c\" rel=\"noreferrer noopener\" target=\"_blank\">github gist with 90 rows<\/a>.<\/p>\n\n\n\n<hr class=\"wp-block-separator\"\/>\n\n\n\n<p>3. The&nbsp;<strong>execution&nbsp;<\/strong>of the test case is just one line of code. You shall have your working directory in the directory with the XML file and&nbsp;<code>my_function<\/code>&nbsp;shall be defined in the global environment.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">RTest.execute(getwd(),\"RTest_medium.xml\")<\/pre>\n\n\n\n<hr class=\"wp-block-separator\"\/>\n\n\n\n<p>4. The<strong>&nbsp;test report<\/strong>&nbsp;now contains one successful and one failed test. Both will be visualized:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/cdn-images-1.medium.com\/max\/800\/1*5YAaVnN0GndEzpjJMo_Ksw.png\" alt=\"\"\/><figcaption>General test outcome in RTest test&nbsp;report<\/figcaption><\/figure>\n\n\n\n<p>additional information is given on all tests. For the test that failed we caused it by setting the sum to be 3.5 instead of 3. It\u2019s reported at the end of the table:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/cdn-images-1.medium.com\/max\/800\/1*FzWodtDBk0Yf22Wym904ig.png\" alt=\"\"\/><figcaption>example of a failed data frame comparison in&nbsp;RTest<\/figcaption><\/figure>\n\n\n\n<p>Moreover the Report contains information on the environment where the test ran:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/cdn-images-1.medium.com\/max\/800\/1*3fXegfECiOojX87PUgQRPg.png\" alt=\"\"\/><figcaption>System information for an RTest test&nbsp;report<\/figcaption><\/figure>\n\n\n\n<p>That\u2019s it. Now you can test any package with RTest.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The specflow and cucumber.io for R. Enabling non-coders to interpret test reports for R-packages, moreover allowing non-coders to create test cases. A step towards simple r package validation. Table of&nbsp;contents [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":4395,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[1],"tags":[413,384,394,381,412],"_links":{"self":[{"href":"https:\/\/engel-wolf.com\/index.php?rest_route=\/wp\/v2\/posts\/4394"}],"collection":[{"href":"https:\/\/engel-wolf.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/engel-wolf.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/engel-wolf.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/engel-wolf.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4394"}],"version-history":[{"count":1,"href":"https:\/\/engel-wolf.com\/index.php?rest_route=\/wp\/v2\/posts\/4394\/revisions"}],"predecessor-version":[{"id":4397,"href":"https:\/\/engel-wolf.com\/index.php?rest_route=\/wp\/v2\/posts\/4394\/revisions\/4397"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/engel-wolf.com\/index.php?rest_route=\/wp\/v2\/media\/4395"}],"wp:attachment":[{"href":"https:\/\/engel-wolf.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4394"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/engel-wolf.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4394"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/engel-wolf.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4394"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}