dhis2-devs team mailing list archive

[Bob Jolliffe <bobjolliffe@xxxxxxxxx>]

Works very nicely.  Just posted a file (dvset.xml):

<dataValueSet xmlns="http://dhis2.org/schema/dataValueSet/0.1";
    dataSet="EEEE2762-6432-4C9A-A739-8E7F1D74F60F"
    period="2011"
    orgUnit="59AC3C28-DDCE-435C-8695-8E457127DADC"
    storedBy="Bob">
  <dataValue dataElement="B01190B8-808C-42C4-AA6F-3F9CF51DC44F" >34</dataValue>
</dataValueSet>

using

curl http://admin:district@xxxxxxxxxxx/dev/api/rpc/dataValueSets -H
"Content-Type: application/xml" -d @dvset.xml

Simple validation seems to work ok.  I get an "Aw, Snap! ..." when
posting twice with the same period but that is probably something you
are not catching yet.

On 15 February 2011 10:14, Jo Størset <storset@xxxxxxxxx> wrote:
> Hi,
>
> just commited a spike for simple(?) external posting of data values. This is a tricky area to design, so I'm hoping this can generate some discussions around it.
>
> The immediate use case for this is an external server doing doing the data collection (mobile based), and then needing an easy way to post dataValueSets, with what I think is more detailed automated validation and feedback than the imp/exp would allow for..
>
> You should be able to try it out by going to [1] (or correspondingly to your local instance) and follow the instructions on screen.
>
> I've tried to promote dataSet and the corresponding dataValueSet as top level concepts, as I think this makes more sense than having isolated data values..
>
> Also, I've created representation for dataValueSet, that is sort of dxf-like, but doesn't really make for the batch-oriented import/export dxf/sdmx is intended for.
> It is possible to tweak the format to be more batch-friendly, but I'm not convinced the use cases for this vs. batch are really similar enough that it makes sense at the cost of simplicity.

I don't agree with this.  If we have to create two separate xml
dialects to handle what you term batch and non-batch (?) then we
really have things wrong.  So we do need to harmonize this.  Having
said that I am very happy to leave dxf v1 where it stands and look to
v2 which should be well enough designed to handle a variety of
scenarios..

Your use of DataValueSet here is very welcome - as you know I have
been advocating this for a while.  Would be nice also to persist it to
provide audit (and simplify dtavalue store) but that is maybe too much
for now.

We would need to think about uuids - particularly with larger
payloads, but otherwise what you are doing here seems to be the
correct way to go.  perhaps with the advent of stable identifiers, we
might have an optional uuid vs id tag to identify the items.

Looking good ....

Cheers
Bob

> One of the things this solution allows for, btw, is use of json instead of xml.
>
> I think we do want an api more in this direction than the current import/export stuff will allow for, but I'm not in any way sure this is the right way to start off. In other words, maybe going with a layer above a reworked version of the xsl/xml based imp/exp framework would be better overall?
>
> Anyway, have a quick look and feed back :)
> Jo
>
> [1] http://dhis.uio.no/dev/api/rpc
>
>
> Den 15. feb. 2011 kl. 12.29 skrev noreply@xxxxxxxxxxxxx:
>
>> ------------------------------------------------------------
>> revno: 2851
>> committer: Jo Størset <storset@xxxxxxxxx>
>> branch nick: dhis2
>> timestamp: Tue 2011-02-15 12:23:26 +0530
>> message:
>>  Added spike for storing dataValueSets through a simple http post (see <dhis-root-url>/api/rpc)
>> added:
>>  dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java
>>  dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/
>>  dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java
>> modified:
>>  dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml
>>
>>
>> --
>> lp:dhis2
>> https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk
>>
>> Your team DHIS 2 developers is subscribed to branch lp:dhis2.
>> To unsubscribe from this branch go to https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
>> === added file 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java'
>> --- dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java      1970-01-01 00:00:00 +0000
>> +++ dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/mapping/IllegalArgumentExceptionMapper.java      2011-02-15 06:53:26 +0000
>> @@ -0,0 +1,23 @@
>> +package org.hisp.dhis.web.api.mapping;
>> +
>> +import javax.ws.rs.core.MediaType;
>> +import javax.ws.rs.core.Response;
>> +import javax.ws.rs.core.Response.Status;
>> +import javax.ws.rs.ext.ExceptionMapper;
>> +import javax.ws.rs.ext.Provider;
>> +
>> +import com.sun.jersey.spi.resource.Singleton;
>> +
>> +@Provider
>> +@Singleton
>> +public class IllegalArgumentExceptionMapper
>> +    implements ExceptionMapper<IllegalArgumentException>
>> +{
>> +
>> +    @Override
>> +    public Response toResponse( IllegalArgumentException e )
>> +    {
>> +        return Response.status( Status.CONFLICT ).entity( "Problem with input: " + e.getMessage() ).type( MediaType.TEXT_PLAIN ).build();
>> +    }
>> +
>> +}
>>
>> === added directory 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc'
>> === added file 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java'
>> --- dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java     1970-01-01 00:00:00 +0000
>> +++ dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/web/api/rpc/RPCResource.java     2011-02-15 06:53:26 +0000
>> @@ -0,0 +1,189 @@
>> +package org.hisp.dhis.web.api.rpc;
>> +
>> +import java.net.URI;
>> +import java.util.List;
>> +import java.util.Set;
>> +
>> +import javax.ws.rs.Consumes;
>> +import javax.ws.rs.GET;
>> +import javax.ws.rs.POST;
>> +import javax.ws.rs.Path;
>> +import javax.ws.rs.PathParam;
>> +import javax.ws.rs.Produces;
>> +import javax.ws.rs.core.Context;
>> +import javax.ws.rs.core.MediaType;
>> +import javax.ws.rs.core.Response;
>> +import javax.ws.rs.core.Response.Status;
>> +import javax.ws.rs.core.UriInfo;
>> +
>> +import org.hisp.dhis.dataelement.DataElement;
>> +import org.hisp.dhis.dataelement.DataElementCategoryOptionCombo;
>> +import org.hisp.dhis.dataset.DataSet;
>> +import org.hisp.dhis.dataset.DataSetService;
>> +import org.hisp.dhis.datavalue.DataValue;
>> +import org.hisp.dhis.datavalue.DataValueService;
>> +import org.hisp.dhis.importexport.datavalueset.DataValueSet;
>> +import org.hisp.dhis.importexport.datavalueset.DataValueSetMapper;
>> +import org.hisp.dhis.organisationunit.OrganisationUnit;
>> +import org.springframework.beans.factory.annotation.Required;
>> +
>> +import com.ibatis.common.logging.Log;
>> +import com.ibatis.common.logging.LogFactory;
>> +
>> +@Path( "/rpc" )
>> +public class RPCResource
>> +{
>> +    private static Log log = LogFactory.getLog( RPCResource.class );
>> +
>> +    private DataValueSetMapper dataValueSetMapper;
>> +
>> +    private DataValueService dataValueService;
>> +
>> +    private DataSetService dataSetService;
>> +
>> +    @Context
>> +    UriInfo uriInfo;
>> +
>> +    @POST
>> +    @Path( "dataValueSets" )
>> +    @Consumes( MediaType.APPLICATION_XML )
>> +    public void storeDataValueSet( DataValueSet dataValueSet )
>> +    {
>> +        List<DataValue> dataValues = dataValueSetMapper.getDataValues( dataValueSet );
>> +
>> +        for ( DataValue dataValue : dataValues )
>> +        {
>> +            dataValueService.addDataValue( dataValue );
>> +        }
>> +    }
>> +
>> +    @GET
>> +    @Produces( MediaType.TEXT_HTML )
>> +    @Path( "dataSets" )
>> +    public String getDataValueSets()
>> +    {
>> +        return getDataValueSet();
>> +    }
>> +
>> +    @GET
>> +    @Produces( MediaType.TEXT_HTML )
>> +    public String getDataValueSet()
>> +    {
>> +        StringBuilder t = new StringBuilder();
>> +        t.append( head( "Data sets available for reporting" ) );
>> +
>> +        t.append( "<h2>Data sets available for reporting</h2>\n<ul>\n" );
>> +        for ( DataSet dataSet : dataSetService.getAllDataSets() )
>> +        {
>> +            URI uri = uriInfo.getBaseUriBuilder().path( "rpc/dataSets/{uuid}" ).build( dataSet.getUuid() );
>> +            t.append( "<li>" ).append( "<a href=\"" ).append( uri ).append( "\">" ).append( dataSet.getName() )
>> +                .append( "</a></li>\n" );
>> +        }
>> +        xmlTemplate( t );
>> +        t.append( tail() );
>> +
>> +        return t.toString();
>> +    }
>> +
>> +    @GET
>> +    @Path( "dataSets/{uuid}" )
>> +    @Produces( MediaType.TEXT_HTML )
>> +    public String getDataSet( @PathParam( "uuid" ) String uuid )
>> +    {
>> +
>> +        DataSet dataSet = dataSetService.getDataSet( uuid );
>> +
>> +        if ( dataSet == null )
>> +        {
>> +            throw new IllegalArgumentException( "No dataset with uuid " + uuid );
>> +        }
>> +
>> +        StringBuilder t = new StringBuilder();
>> +
>> +        t.append( head( "Data set " + dataSet.getName() ) );
>> +        t.append( "<p>Uuid: " ).append( dataSet.getUuid() ).append( "<br>\n" );
>> +        t.append( "Period type: " ).append( dataSet.getPeriodType().getName() ).append( " - " )
>> +            .append( dataSet.getPeriodType().getIsoFormat() );
>> +        t.append( "</p>\n" );
>> +
>> +        t.append( "<h2>Org units reporting data set</h2>\n<ul>" );
>> +        for ( OrganisationUnit unit : dataSet.getOrganisationUnis() )
>> +        {
>> +            t.append( "<li><b>" ).append( unit.getName() ).append( "</b> - " ).append( unit.getUuid() )
>> +                .append( "</li>" );
>> +        }
>> +        t.append( "</ul>\n" );
>> +
>> +        t.append( "<h2>Data elements in data set</h2>\n<ul>" );
>> +        for ( DataElement element : dataSet.getDataElements() )
>> +        {
>> +            t.append( "<li><b>" ).append( element.getName() ).append( "</b> (" ).append( element.getType() )
>> +                .append( ") - " ).append( element.getUuid() );
>> +
>> +            Set<DataElementCategoryOptionCombo> optionCombos = element.getCategoryCombo().getOptionCombos();
>> +            if ( optionCombos.size() > 1 )
>> +            {
>> +                t.append( "<br>CategoryOptionCombos\n<ul>\n" );
>> +                for ( DataElementCategoryOptionCombo optionCombo : optionCombos )
>> +                {
>> +                    t.append( "<li><b>" ).append( optionCombo.getName() ).append( "</b> - " )
>> +                        .append( optionCombo.getUuid() ).append( "</li>" );
>> +                }
>> +                t.append( "</ul>\n" );
>> +            }
>> +            t.append( "</li>\n" );
>> +        }
>> +        t.append( "</ul>" );
>> +        xmlTemplate( t );
>> +        t.append( tail() );
>> +
>> +        return t.toString();
>> +    }
>> +
>> +    private String head( String title )
>> +    {
>> +        return "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\" \"http://www.w3.org/TR/html4/strict.dtd\";> \n<html><head><title>"
>> +            + title + "</title></head>\n" + "<body>\n<h1>" + title + "</h1>\n";
>> +    }
>> +
>> +    private void xmlTemplate( StringBuilder t )
>> +    {
>> +        t.append( "<h2>Xml template</h2>\n" );
>> +
>> +        URI uri = uriInfo.getBaseUriBuilder().path( "rpc/dataValueSets" ).build();
>> +        t.append( "<p>Post according to the following template to " );
>> +        t.append( "<a href=\"" ).append( uri ).append( "\">" ).append( uri ).append( "</a>:</p>" );
>> +
>> +        t.append( "<pre>" ).append( "&lt;dataValueSet xmlns=\"http://dhis2.org/schema/dataValueSet/0.1\"\n"; );
>> +        t.append( "    dataSet=\"dataSet UUID\" \n    period=\"periodInIsoFormat\"\n    orgUnit=\"unit UUID\"" );
>> +        t.append( "\n    storedBy=\"user\"&gt;" );
>> +
>> +        t.append( "\n  &lt;dataValue dataElement=\"data element UUID\" categoryOptionCombo=\"UUID, only specify if used\" &gt;value&lt;/dataValue&gt;" );
>> +        t.append( "\n&lt;/dataValueSet&gt;" );
>> +        t.append( "</pre>" );
>> +    }
>> +
>> +    private String tail()
>> +    {
>> +        return "</body>\n</html>\n";
>> +    }
>> +
>> +    @Required
>> +    public void setDataValueSetMapper( DataValueSetMapper dataValueSetMapper )
>> +    {
>> +        this.dataValueSetMapper = dataValueSetMapper;
>> +    }
>> +
>> +    @Required
>> +    public void setDataValueService( DataValueService dataValueService )
>> +    {
>> +        this.dataValueService = dataValueService;
>> +    }
>> +
>> +    @Required
>> +    public void setDataSetService( DataSetService dataSetService )
>> +    {
>> +        this.dataSetService = dataSetService;
>> +    }
>> +
>> +}
>>
>> === modified file 'dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml'
>> --- dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml   2011-01-21 18:28:26 +0000
>> +++ dhis-2/dhis-web/dhis-web-api/src/main/resources/META-INF/dhis/beans.xml   2011-02-15 06:53:26 +0000
>> @@ -60,6 +60,9 @@
>>   <bean id="org.hisp.dhis.web.api.mapping.NotAllowedExceptionMapper" class="org.hisp.dhis.web.api.mapping.NotAllowedExceptionMapper"
>>     scope="singleton" />
>>
>> +  <bean id="org.hisp.dhis.web.api.mapping.IllegalArgumentExceptionMapper" class="org.hisp.dhis.web.api.mapping.IllegalArgumentExceptionMapper"
>> +    scope="singleton" />
>> +
>>   <bean id="JacksonJaxbJsonProvider" class="org.codehaus.jackson.jaxrs.JacksonJaxbJsonProvider" scope="singleton" />
>>
>>   <!-- ImportDataValue beans -->
>> @@ -76,6 +79,13 @@
>>   <bean id="org.hisp.dhis.web.api.service.ModelMapping" class="org.hisp.dhis.web.api.service.ModelMapping">
>>     <property name="categoryService" ref="org.hisp.dhis.dataelement.DataElementCategoryService" />
>>   </bean>
>> +
>> +
>> +  <bean id="org.hisp.dhis.web.api.rpc.RPCResource" class="org.hisp.dhis.web.api.rpc.RPCResource">
>> +    <property name="dataValueService" ref="org.hisp.dhis.datavalue.DataValueService" />
>> +    <property name="dataValueSetMapper" ref="org.hisp.dhis.importexport.datavalueset.DataValueSetMapper" />
>> +    <property name="dataSetService" ref="org.hisp.dhis.dataset.DataSetService" />
>> +  </bean>
>>
>> </beans>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>