{% extends "base.html" %}

{% block content %}
<div class="section">
<h2>ShExStatements: Documentation</h2>

<p>ShExStatements allows users to generate shape expressions from simple CSV statements or files. <code>shexstatements</code> can be used from the command line or can be accessed from a browser.</p>

<h2>Objectives</h2>

<ul>
<li>Easily generate shape expressions (ShEx) from CSV files </li>
<li>Simple syntax, with 5 columns
<ul>
<li>Node name</li>
<li>Property</li>
<li>Allowed values</li>
<li>Cardinality (optional)</li>
<li>Comments (optional)</li>
</ul></li>
</ul>

<h2>Setup</h2>

<p>Clone the <strong>ShExStatements</strong> repository.
<pre>
<code>
$ git clone https://github.com/johnsamuelwrites/ShExStatements.git
</code>
</pre>
</p>

<p>Go to <strong>ShExStatements</strong> directory.</p>
<pre>
<code>
$ cd ShExStatements
</code>
</pre>

<h3>Virtual Environment</h3>
Install modules required by <b>ShExStatements</b> (here: installing into a virtual environment).
<pre>
<code>
$ python3 -m venv .venv
$ source ./.venv/bin/activate
$ pip3 install .
</code>
</pre>


<p>Consider an example CSV file <code>language.csv</code> in the folder <code>examples/</code>.
The file contains an example description of a language on Wikidata. This file uses comma as a delimiter to separate the values.</p>
<pre>
<code>
wd,&lt;http://www.wikidata.org/entity/&gt;,,,
wdt,&lt;http://www.wikidata.org/prop/direct/&gt;,,,
xsd,&lt;http://www.w3.org/2001/XMLSchema#&gt;,,,

@language,wdt:P31,wd:Q34770,,# instance of a language
@language,wdt:P1705,LITERAL,,# native name
@language,wdt:P17,.,+,# spoken in country
@language,wdt:P2989,.,+,# grammatical cases
@language,wdt:P282,.,+,# writing system
@language,wdt:P1098,.,+,# speakers
@language,wdt:P1999,.,*,# UNESCO language status
@language,wdt:P2341,.,+,# indigenous to
</code>
</pre>

<p>There are five columns in the CSV file. </p>
<ul>
  <li>Column 1 is used for specifying the node name, starting with @.</li>
  <li>Column 2 for specifying the property value  </li>
  <li>Column 3 for a set of possible values  </li>
  <li>Column 4 for cardinality (+,*)  </li>
  <li>Column 5 for comments. Comments start with #.  </li>
</ul>

<p>Columns 1, 2, 3 are mandatory. Column 3 can be a special value like . (period to say 'any' value). The first three lines in the above file are used for specifying the prefixes. In this case, columns 3,4 and 5 are empty. </p>

<p>Cardinality can be any one of the following values</p>
<ul>
  <li> * : zero or more values  </li>
  <li> + : one or more values  </li>
  <li> m : m number of values  </li>
  <li> m,n : any number of values between m and n (including m and n).  </li>
</ul>

<p>Run the following command for the above file.</p>
<pre>
<code>
$ ./shexstatements.sh examples/language.csv
</code>
</pre>
</p>


<p>The shape expression generated by <strong>ShExStatements</strong> will look like</p>
<pre>
<code>
PREFIX xsd: &lt;http://www.w3.org/2001/XMLSchema#&gt;
PREFIX wd: &lt;http://www.wikidata.org/entity/&gt;
PREFIX wdt: &lt;http://www.wikidata.org/prop/direct/&gt;
start = @&lt;language&gt;
&lt;language&gt; {
  wdt:P31 [ wd:Q34770  ] ;# instance of a language
  wdt:P1705 LITERAL ;# native name
  wdt:P17 .+ ;# spoken in country
  wdt:P2989 .+ ;# grammatical cases
  wdt:P282 .+ ;# writing system
  wdt:P1098 .+ ;# speakers
  wdt:P1999 .* ;# UNESCO language status
  wdt:P2341 .+ ;# indigenous to
}
</code>
</pre>

<p>CSV file can use delimiters like semicolon (;). Take for example, the following command works with a file using semi-colon as a delimiter. </p>

<p>
<pre>
<code>
$ ./shexstatements.sh examples/languagedelimsemicolon.csv --delim ";"
</code>
</pre>
</p>

<p>But sometimes, users may like to specify the header. In that case, they can make use of <code>-s</code> or <code>--skipheader</code> to tell the generator to skip the header (firsrt line of CSV).</p>

<pre>
<code>
$ ./shexstatements.sh --skipheader examples/languageheader.csv
</code>
</pre>

<h3>Web interface</h3>
<p><code>shexstatements</code> can also be accessed from a web interface. Clone the <b>ShExStatements</b> repository.</p>
<pre>
<code>
$ git clone https://github.com/johnsamuelwrites/ShExStatements.git
</code>
</pre>

<p>Go to <b>ShExStatements</b> directory.</p>
<pre>
<code>
$ cd ShExStatements
</code>
</pre>

<p>Install modules required by <strong>ShExStatements</strong> (here: installing into a virtual environment).</p>
<pre>
<code>
$ python3 -m venv .venv
$ source ./.venv/bin/activate
$ pip3 install .
</code>
</pre>

<p>Now run the application.</p>
<pre>
<code>
$ ./shexstatements.sh -r  
</code>
</pre>
<p>Check the URL <code>http://127.0.0.1:5000/</code>.</p>


<h2>ShExJ</h2>
<p>Use <code>-j</code> or <code>--shexj</code> to generate ShEx JSON Syntax (ShExJ) instead of default ShEx Compact syntax (ShExC).</p>

<p>
<pre>
<code>
$ ./shexstatements.sh --shexj examples/language.csv
</code>
</pre>
</p>

<p>The output will be similiar to:</p>

<p>
<pre>
<code>
{
  "type": "Schema",
  "start": "language",
  "shapes": [
    {
      "type": "Shape",
      "id": "language",
      "expression": {
        "type": "EachOf",
        "expressions": [
          {
            "type": "TripleConstraint",
            "predicate": "http://www.wikidata.org/prop/direct/P31",
            "valueExpr": {
              "type": "NodeConstraint",
              "values": [
                "http://www.wikidata.org/entity/Q34770"
              ]
            }
          },
          {
            "type": "TripleConstraint",
            "predicate": "http://www.wikidata.org/prop/direct/P1705",
            "valueExpr": {
              "type": "NodeConstraint",
              "nodeKind": "literal"
            }
          },
          {
            "type": "TripleConstraint",
            "predicate": "http://www.wikidata.org/prop/direct/P17",
            "min": 1,
            "max": -1
          },
          {
            "type": "TripleConstraint",
            "predicate": "http://www.wikidata.org/prop/direct/P2989",
            "min": 1,
            "max": -1
          },
          {
            "type": "TripleConstraint",
            "predicate": "http://www.wikidata.org/prop/direct/P282",
            "min": 1,
            "max": -1
          },
          {
            "type": "TripleConstraint",
            "predicate": "http://www.wikidata.org/prop/direct/P1098",
            "min": 1,
            "max": -1
          },
          {
            "type": "TripleConstraint",
            "predicate": "http://www.wikidata.org/prop/direct/P1999",
            "min": 0,
            "max": -1
          },
          {
            "type": "TripleConstraint",
            "predicate": "http://www.wikidata.org/prop/direct/P2341",
            "min": 1,
            "max": -1
          }
        ]
      }
    }
  ]
}
</code>
</pre>
It's also possible to use application profiles of the following form
<pre>
<code>
Entity_name,Property,Property_label,Mand,Repeat,Value,Value_type,Annotation
</code>
</pre>
and Shape expressions can be generated using the following form
<pre>
<code>
$ ./shexstatements.sh -ap --skipheader examples/languageap.csv
</code>
</pre>
</p>
        <h2>Test cases and Code Coverage</h2>
        
        <p>All the test cases can be run in  the following manner.</p>
        <pre>
          <code>
$ python3 -m tests.tests
          </code>
        </pre>
Code coverage report can also be generated by running the unit tests using the coverage tool.
        <pre>
          <code>
$ coverage run --source=shexstatements -m unittest tests.test
$ coverage report -m
          </code>
        </pre>

        <h2>Examples</h2>
<p>There are example CSV files in the <a href="https://github.com/johnsamuelwrites/ShExStatements/tree/master/examples">examples</a> folder.</p>
</div>
{% endblock %}