dhis2-devs team mailing list archive
-
dhis2-devs team
-
Mailing list archive
-
Message #04262
Re: [Branch ~dhis2-devs-core/dhis2/trunk] Rev 1404: Implemented DHIS 1.4 export of data values
2010/2/8 Bob Jolliffe <bobjolliffe@xxxxxxxxx>
> 2010/2/8 Lars Helge Øverland <larshelge@xxxxxxxxx>:
> >
> >
> > On Mon, Feb 8, 2010 at 11:59 AM, <noreply@xxxxxxxxxxxxx> wrote:
> >>
> >> ------------------------------------------------------------
> >> revno: 1404
> >> committer: Lars Helge Oeverland <larshelge@xxxxxxxxx>
> >> branch nick: trunk
> >> timestamp: Mon 2010-02-08 11:56:08 +0100
> >> message:
> >> Implemented DHIS 1.4 export of data values
> >> modified:
> >
> >
> > There is actually still a snag here. DHIS 1.4 only accepts the 7zip/lzma
> > format currently. I have talked Greg into making 1.4 accept zip/deflate
> too.
>
> Isn't it better to have gzip/deflate? We are not talking about
> compressing an archive of files but rather compressing a single
> stream.
>
> Also I think the original "void write( BufferedWriter writer,
> ExportParams params )" might be better than "void write(
> ZipOutputStream out, ExportParams params )". It seems unnecessary
> that the convertor should have to know anything about zip, gzip, 7zip
> or what have you. It should know how to write csv to a stream.
>
> I suspect that how the stream is later compressed is better deferred to
> later.
>
Problem is that DHIS 1.4 uses a CSV file for data and an XML file for
meta-data and keeps both inside the archive... I was was not able to have
multiple zip entries while using the Writer..
>
> Cheers
> Bob
>
> >
> >>
> >>
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/CSVConverter.java
> >>
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/ExportPipeThread.java
> >>
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/converter/ReportTableDataConverter.java
> >>
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/CSVExportPipeThread.java
> >>
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/DefaultCSVExportService.java
> >>
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/util/CsvUtil.java
> >>
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/converter/DataValueConverter.java
> >>
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/exporter/DefaultDhis14XMLExportService.java
> >>
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/resources/META-INF/dhis/beans.xml
> >>
> >>
> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/DateUtils.java
> >>
> >>
> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/StreamUtils.java
> >>
> >>
> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties
> >>
> >>
> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/externalExportMenu.vm
> >>
> >>
> >> --
> >> lp:dhis2
> >> https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk
> >>
> >> Your team DHIS 2 developers is subscribed to branch lp:dhis2.
> >> To unsubscribe from this branch go to
> >>
> https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
> .
> >>
> >> === modified file
> >>
> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/CSVConverter.java'
> >> ---
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/CSVConverter.java
> >> 2009-03-03 16:46:36 +0000
> >> +++
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/CSVConverter.java
> >> 2010-02-08 10:56:08 +0000
> >> @@ -28,7 +28,7 @@
> >> */
> >>
> >> import java.io.BufferedReader;
> >> -import java.io.BufferedWriter;
> >> +import java.util.zip.ZipOutputStream;
> >>
> >> /**
> >> * @author Lars Helge Overland
> >> @@ -36,7 +36,7 @@
> >> */
> >> public interface CSVConverter
> >> {
> >> - void write( BufferedWriter writer, ExportParams params );
> >> + void write( ZipOutputStream out, ExportParams params );
> >>
> >> void read( BufferedReader reader, ImportParams params );
> >> }
> >>
> >> === modified file
> >>
> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/ExportPipeThread.java'
> >> ---
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/ExportPipeThread.java
> >> 2009-11-02 15:55:44 +0000
> >> +++
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/ExportPipeThread.java
> >> 2010-02-08 10:56:08 +0000
> >> @@ -52,7 +52,8 @@
> >>
> >> private List<XMLConverter> xsdConverters = new
> >> ArrayList<XMLConverter>();
> >> private List<XMLConverter> xmlConverters = new
> >> ArrayList<XMLConverter>();
> >> -
> >> + private List<CSVConverter> csvConverters = new
> >> ArrayList<CSVConverter>();
> >> +
> >> //
> >>
> -------------------------------------------------------------------------
> >> // Parameters
> >> //
> >>
> -------------------------------------------------------------------------
> >> @@ -122,6 +123,11 @@
> >> this.xmlConverters.add( converter );
> >> }
> >>
> >> + public void registerCSVConverter ( CSVConverter converter )
> >> + {
> >> + this.csvConverters.add( converter );
> >> + }
> >> +
> >> //
> >>
> -------------------------------------------------------------------------
> >> // Thread implementation
> >> //
> >>
> -------------------------------------------------------------------------
> >> @@ -159,9 +165,20 @@
> >> }
> >>
> >> afterXML( writer );
> >> -
> >> +
> >> closeDocument( writer );
> >> -
> >> +
> >> + StreamUtils.closeZipEntry( zipOutputStream );
> >> +
> >> + //
> >> -----------------------------------------------------------------
> >> + // CSV
> >> + //
> >> -----------------------------------------------------------------
> >> +
> >> + for ( CSVConverter converter : csvConverters )
> >> + {
> >> + converter.write( zipOutputStream, params );
> >> + }
> >> +
> >> log.info( "Export done" );
> >> }
> >> catch ( Exception ex )
> >> @@ -172,12 +189,10 @@
> >> }
> >> finally
> >> {
> >> - StreamUtils.finishZipEntry( zipOutputStream );
> >> + writer.closeWriter();
> >>
> >> StreamUtils.closeOutputStream( zipOutputStream );
> >>
> >> - writer.closeWriter();
> >> -
> >> NameMappingUtil.clearMapping();
> >> }
> >> }
> >>
> >> === modified file
> >>
> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/converter/ReportTableDataConverter.java'
> >> ---
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/converter/ReportTableDataConverter.java
> >> 2009-06-10 22:25:07 +0000
> >> +++
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/converter/ReportTableDataConverter.java
> >> 2010-02-08 10:56:08 +0000
> >> @@ -27,14 +27,17 @@
> >> * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> >> */
> >>
> >> -import static org.hisp.dhis.importexport.csv.util.CsvUtil.SEPARATOR;
> >> +import static
> org.hisp.dhis.importexport.csv.util.CsvUtil.CSV_EXTENSION;
> >> +import static org.hisp.dhis.importexport.csv.util.CsvUtil.NEWLINE;
> >> +import static org.hisp.dhis.importexport.csv.util.CsvUtil.SEPARATOR_B;
> >> import static org.hisp.dhis.importexport.csv.util.CsvUtil.csvEncode;
> >>
> >> import java.io.BufferedReader;
> >> -import java.io.BufferedWriter;
> >> import java.io.IOException;
> >> import java.util.Iterator;
> >> import java.util.SortedMap;
> >> +import java.util.zip.ZipEntry;
> >> +import java.util.zip.ZipOutputStream;
> >>
> >> import org.hisp.dhis.importexport.CSVConverter;
> >> import org.hisp.dhis.importexport.ExportParams;
> >> @@ -69,27 +72,29 @@
> >> // CSVConverter implementation
> >> //
> >>
> -------------------------------------------------------------------------
> >>
> >> - public void write( BufferedWriter writer, ExportParams params )
> >> + public void write( ZipOutputStream out, ExportParams params )
> >> {
> >> try
> >> {
> >> - for ( Integer id : params.getReportTables() ) //TODO more
> >> than one?
> >> + for ( Integer id : params.getReportTables() )
> >> {
> >> + out.putNextEntry( new ZipEntry( "ReportTable" + id +
> >> CSV_EXTENSION ) );
> >> +
> >> ReportTableData data =
> >> reportTableService.getReportTableData( id, params.getFormat() );
> >>
> >> Iterator<String> columns =
> >> data.getPrettyPrintColumns().iterator();
> >>
> >> while ( columns.hasNext() )
> >> {
> >> - writer.write( csvEncode( columns.next() ) );
> >> + out.write( csvEncode( columns.next() ).getBytes()
> );
> >>
> >> if ( columns.hasNext() )
> >> {
> >> - writer.write( SEPARATOR );
> >> + out.write( SEPARATOR_B );
> >> }
> >> }
> >>
> >> - writer.newLine();
> >> + out.write( NEWLINE );
> >>
> >> for ( SortedMap<Integer, String> row : data.getRows() )
> >> {
> >> @@ -97,15 +102,15 @@
> >>
> >> while ( values.hasNext() )
> >> {
> >> - writer.write( csvEncode( values.next() ) );
> >> + out.write( csvEncode( values.next()
> ).getBytes()
> >> );
> >>
> >> if ( values.hasNext() )
> >> {
> >> - writer.write( SEPARATOR );
> >> + out.write( SEPARATOR_B );
> >> }
> >> }
> >>
> >> - writer.newLine();
> >> + out.write( NEWLINE );
> >> }
> >> }
> >> }
> >>
> >> === modified file
> >>
> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/CSVExportPipeThread.java'
> >> ---
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/CSVExportPipeThread.java
> >> 2009-06-10 22:25:07 +0000
> >> +++
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/CSVExportPipeThread.java
> >> 2010-02-08 10:56:08 +0000
> >> @@ -27,9 +27,9 @@
> >> * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> >> */
> >>
> >> -import java.io.BufferedWriter;
> >> import java.util.ArrayList;
> >> import java.util.List;
> >> +import java.util.zip.ZipOutputStream;
> >>
> >> import org.apache.commons.logging.Log;
> >> import org.apache.commons.logging.LogFactory;
> >> @@ -48,13 +48,6 @@
> >> {
> >> private static final Log log = LogFactory.getLog(
> >> CSVExportPipeThread.class );
> >>
> >> - private BufferedWriter writer;
> >> -
> >> - public void setWriter( BufferedWriter writer )
> >> - {
> >> - this.writer = writer;
> >> - }
> >> -
> >> private ExportParams params;
> >>
> >> public void setParams( ExportParams params )
> >> @@ -62,6 +55,13 @@
> >> this.params = params;
> >> }
> >>
> >> + private ZipOutputStream outputStream;
> >> +
> >> + public void setOutputStream( ZipOutputStream outputStream )
> >> + {
> >> + this.outputStream = outputStream;
> >> + }
> >> +
> >> private List<CSVConverter> converters = new
> ArrayList<CSVConverter>();
> >>
> >> public void registerCSVConverter( CSVConverter converter )
> >> @@ -90,14 +90,18 @@
> >>
> >> for ( CSVConverter converter : converters )
> >> {
> >> - converter.write( writer, params );
> >> + converter.write( outputStream, params );
> >> }
> >>
> >> log.info( "Export finished" );
> >> }
> >> + catch ( Exception ex )
> >> + {
> >> + throw new RuntimeException( ex );
> >> + }
> >> finally
> >> {
> >> - StreamUtils.closeWriter( writer );
> >> + StreamUtils.closeOutputStream( outputStream );
> >> }
> >> }
> >> }
> >>
> >> === modified file
> >>
> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/DefaultCSVExportService.java'
> >> ---
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/DefaultCSVExportService.java
> >> 2009-06-10 22:25:07 +0000
> >> +++
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/DefaultCSVExportService.java
> >> 2010-02-08 10:56:08 +0000
> >> @@ -28,13 +28,11 @@
> >> */
> >>
> >> import java.io.BufferedInputStream;
> >> -import java.io.BufferedWriter;
> >> +import java.io.BufferedOutputStream;
> >> import java.io.IOException;
> >> import java.io.InputStream;
> >> -import java.io.OutputStreamWriter;
> >> import java.io.PipedInputStream;
> >> import java.io.PipedOutputStream;
> >> -import java.util.zip.ZipEntry;
> >> import java.util.zip.ZipOutputStream;
> >>
> >> import org.hibernate.SessionFactory;
> >> @@ -50,8 +48,6 @@
> >> public class DefaultCSVExportService
> >> implements ExportService
> >> {
> >> - private static final String ZIP_ENTRY_NAME = "Export.csv";
> >> -
> >> //
> >>
> -------------------------------------------------------------------------
> >> // Dependencies
> >> //
> >>
> -------------------------------------------------------------------------
> >> @@ -87,20 +83,18 @@
> >>
> >> PipedInputStream in = new PipedInputStream( out );
> >>
> >> - ZipOutputStream zipOut = new ZipOutputStream( out );
> >> + ZipOutputStream zipOut = new ZipOutputStream( new
> >> BufferedOutputStream( out ) );
> >>
> >> - zipOut.putNextEntry( new ZipEntry( ZIP_ENTRY_NAME ) );
> >> + //zipOut.putNextEntry( new ZipEntry( ZIP_ENTRY_NAME ) );
> >>
> >> - BufferedWriter writer = new BufferedWriter( new
> >> OutputStreamWriter( zipOut ) );
> >> -
> >> //
> >> -----------------------------------------------------------------
> >> // Writes to one end of the pipe
> >> //
> >> -----------------------------------------------------------------
> >>
> >> CSVExportPipeThread thread = new CSVExportPipeThread(
> >> sessionFactory );
> >>
> >> - thread.setWriter( writer );
> >> thread.setParams( params );
> >> + thread.setOutputStream( zipOut );
> >>
> >> thread.registerCSVConverter( new ReportTableDataConverter(
> >> reportTableService ) );
> >>
> >>
> >> === modified file
> >>
> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/util/CsvUtil.java'
> >> ---
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/util/CsvUtil.java
> >> 2009-04-16 09:53:17 +0000
> >> +++
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/util/CsvUtil.java
> >> 2010-02-08 10:56:08 +0000
> >> @@ -33,15 +33,80 @@
> >> */
> >> public class CsvUtil
> >> {
> >> - public static final char SEPARATOR = ',';
> >> -
> >> - private static final String ENCLOSURE = "\"";
> >> -
> >> - public static String csvEncode( String string )
> >> - {
> >> - string = string.replaceAll( ENCLOSURE, ENCLOSURE + ENCLOSURE );
> >> - string = ENCLOSURE + string + ENCLOSURE;
> >> + public static final String SEPARATOR = ",";
> >> + public static final byte[] SEPARATOR_B = SEPARATOR.getBytes();
> >> + public static final byte[] NEWLINE = "\n".getBytes();
> >> +
> >> + public static final String CSV_EXTENSION = ".csv";
> >> + private static final String ENCLOSURE = "\"";
> >> + private static final String EMPTY = "";
> >> +
> >> + /**
> >> + * Encodes the given value to a CSV acceptable value.
> >> + *
> >> + * @param value the value.
> >> + * @return the CSV encoded value.
> >> + */
> >> + public static String csvEncode( int value )
> >> + {
> >> + return csvEncode( String.valueOf( value ) );
> >> + }
> >> +
> >> + /**
> >> + * Encodes the given value to a CSV acceptable value.
> >> + *
> >> + * @param value the value.
> >> + * @return the CSV encoded value.
> >> + */
> >> + public static String csvEncode( String value )
> >> + {
> >> + if ( value == null )
> >> + {
> >> + value = EMPTY;
> >> + }
> >> + else
> >> + {
> >> + value = value.replaceAll( ENCLOSURE, ENCLOSURE + ENCLOSURE
> );
> >> + value = ENCLOSURE + value + ENCLOSURE;
> >> + }
> >> +
> >> + return value;
> >> + }
> >> +
> >> + /**
> >> + * Appends a separator to the value and returns the value as a byte
> >> array.
> >> + *
> >> + * @param value the value.
> >> + * @return a byte araray.
> >> + */
> >> + public static byte[] getCsvValue( int value )
> >> + {
> >> + return getCsvEndValue( value + SEPARATOR );
> >> + }
> >> +
> >> + /**
> >> + * Appends a separator to the value and returns the value as a byte
> >> array.
> >> + *
> >> + * @param value the value.
> >> + * @return a byte araray.
> >> + */
> >> + public static byte[] getCsvValue( String value )
> >> + {
> >> + return getCsvEndValue( value + SEPARATOR );
> >> + }
> >> +
> >> + public static byte[] getCsvEndValue( int value )
> >> + {
> >> + return getCsvEndValue( String.valueOf( value ) );
> >> + }
> >> +
> >> + public static byte[] getCsvEndValue( String value )
> >> + {
> >> + if ( value == null )
> >> + {
> >> + return EMPTY.getBytes();
> >> + }
> >>
> >> - return string;
> >> + return ( value ).getBytes();
> >> }
> >> }
> >>
> >> === modified file
> >>
> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/converter/DataValueConverter.java'
> >> ---
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/converter/DataValueConverter.java
> >> 2010-02-08 07:04:26 +0000
> >> +++
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/converter/DataValueConverter.java
> >> 2010-02-08 10:56:08 +0000
> >> @@ -27,17 +27,28 @@
> >> * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> >> */
> >>
> >> +import static org.hisp.dhis.importexport.csv.util.CsvUtil.NEWLINE;
> >> +import static org.hisp.dhis.importexport.csv.util.CsvUtil.SEPARATOR_B;
> >> +import static org.hisp.dhis.importexport.csv.util.CsvUtil.csvEncode;
> >> +import static org.hisp.dhis.importexport.csv.util.CsvUtil.getCsvValue;
> >> +import static
> org.hisp.dhis.importexport.csv.util.CsvUtil.getCsvEndValue;
> >> +
> >> import java.io.BufferedReader;
> >> -import java.io.BufferedWriter;
> >> import java.io.IOException;
> >> +import java.util.Collection;
> >> import java.util.Map;
> >> +import java.util.zip.ZipEntry;
> >> +import java.util.zip.ZipOutputStream;
> >>
> >> import org.amplecode.quick.BatchHandler;
> >> +import org.amplecode.quick.StatementManager;
> >> import org.hisp.dhis.dataelement.DataElement;
> >> import org.hisp.dhis.dataelement.DataElementCategoryOptionCombo;
> >> import org.hisp.dhis.dataelement.DataElementCategoryService;
> >> +import org.hisp.dhis.datamart.DataMartService;
> >> import org.hisp.dhis.datavalue.DataValue;
> >> import org.hisp.dhis.datavalue.DataValueService;
> >> +import org.hisp.dhis.datavalue.DeflatedDataValue;
> >> import org.hisp.dhis.importexport.CSVConverter;
> >> import org.hisp.dhis.importexport.ExportParams;
> >> import org.hisp.dhis.importexport.GroupMemberType;
> >> @@ -47,7 +58,10 @@
> >> import org.hisp.dhis.importexport.converter.AbstractDataValueConverter;
> >> import org.hisp.dhis.organisationunit.OrganisationUnit;
> >> import org.hisp.dhis.period.Period;
> >> +import org.hisp.dhis.period.PeriodService;
> >> +import org.hisp.dhis.system.util.DateUtils;
> >> import org.hisp.dhis.system.util.MimicingHashMap;
> >> +import org.hisp.dhis.system.util.StreamUtils;
> >>
> >> /**
> >> * @author Lars Helge Overland
> >> @@ -57,8 +71,11 @@
> >> extends AbstractDataValueConverter implements CSVConverter
> >> {
> >> private static final String SEPARATOR = ",";
> >> + private static final String FILENAME = "RoutineData.txt";
> >>
> >> private DataElementCategoryService categoryService;
> >> + private PeriodService periodService;
> >> + private StatementManager statementManager;
> >>
> >> //
> >>
> -------------------------------------------------------------------------
> >> // Properties
> >> @@ -72,6 +89,14 @@
> >> // Constructor
> >> //
> >>
> -------------------------------------------------------------------------
> >>
> >> + public DataValueConverter( PeriodService periodService,
> >> DataMartService dataMartService,
> >> + StatementManager statementManager )
> >> + {
> >> + this.periodService = periodService;
> >> + this.dataMartService = dataMartService;
> >> + this.statementManager = statementManager;
> >> + }
> >> +
> >> /**
> >> * Constructor for read operations.
> >> */
> >> @@ -95,9 +120,81 @@
> >> // CSVConverter implementation
> >> //
> >>
> -------------------------------------------------------------------------
> >>
> >> - public void write( BufferedWriter writer, ExportParams params )
> >> + public void write( ZipOutputStream out, ExportParams params )
> >> {
> >> - // Not implemented
> >> + try
> >> + {
> >> + out.putNextEntry( new ZipEntry( FILENAME ) );
> >> +
> >> + out.write( getCsvValue( csvEncode( "RoutineDataID" ) ) );
> >> + out.write( getCsvValue( csvEncode( "OrgUnitID" ) ) );
> >> + out.write( getCsvValue( csvEncode( "DataElementID" ) ) );
> >> + out.write( getCsvValue( csvEncode( "DataPeriodID" ) ) );
> >> + out.write( getCsvValue( csvEncode( "EntryText" ) ) );
> >> + out.write( getCsvValue( csvEncode( "EntryYesNo" ) ) );
> >> + out.write( getCsvValue( csvEncode( "EntryNumber" ) ) );
> >> + out.write( getCsvValue( csvEncode( "EntryDate" ) ) );
> >> + out.write( getCsvValue( csvEncode( "EntryMemo" ) ) );
> >> + out.write( getCsvValue( csvEncode( "EntryObject" ) ) );
> >> + out.write( getCsvValue( csvEncode( "Check" ) ) );
> >> + out.write( getCsvValue( csvEncode( "Verified" ) ) );
> >> + out.write( getCsvValue( csvEncode( "Deleted" ) ) );
> >> + out.write( getCsvValue( csvEncode( "Comment" ) ) );
> >> + out.write( getCsvValue( csvEncode( "LastUserID" ) ) );
> >> + out.write( getCsvEndValue( csvEncode( "LastUpdated" ) ) );
> >> +
> >> + out.write( NEWLINE );
> >> +
> >> + if ( params.isIncludeDataValues() )
> >> + {
> >> + if ( params.getStartDate() != null &&
> params.getEndDate()
> >> != null )
> >> + {
> >> + Collection<DeflatedDataValue> values = null;
> >> +
> >> + Collection<Period> periods =
> >> periodService.getIntersectingPeriods( params.getStartDate(),
> >> params.getEndDate() );
> >> +
> >> + statementManager.initialise();
> >> +
> >> + for ( final Integer element :
> >> params.getDataElements() )
> >> + {
> >> + for ( final Period period : periods )
> >> + {
> >> + values =
> >> dataMartService.getDeflatedDataValues( element, period.getId(),
> >> params.getOrganisationUnits() );
> >> +
> >> + for ( final DeflatedDataValue value :
> values
> >> )
> >> + {
> >> + out.write( getCsvValue( 0 ) );
> >> + out.write( getCsvValue(
> >> value.getSourceId() ) );
> >> + out.write( getCsvValue(
> >> value.getDataElementId() ) );
> >> + out.write( getCsvValue(
> >> value.getPeriodId() ) );
> >> + out.write( SEPARATOR_B );
> >> + out.write( SEPARATOR_B );
> >> + out.write( getCsvValue( csvEncode(
> >> value.getValue() ) ) );
> >> + out.write( SEPARATOR_B );
> >> + out.write( SEPARATOR_B );
> >> + out.write( SEPARATOR_B );
> >> + out.write( getCsvValue( 0 ) );
> >> + out.write( getCsvValue( 0 ) );
> >> + out.write( getCsvValue( 0 ) );
> >> + out.write( getCsvValue( csvEncode(
> >> value.getComment() ) ) );
> >> + out.write( getCsvValue( 1 ) );
> >> + out.write( getCsvEndValue(
> >> DateUtils.getAccessDateString( value.getTimestamp() ) ) );
> >> +
> >> + out.write( NEWLINE );
> >> + }
> >> + }
> >> + }
> >> +
> >> + statementManager.destroy();
> >> + }
> >> + }
> >> +
> >> + StreamUtils.closeZipEntry( out );
> >> + }
> >> + catch ( IOException ex )
> >> + {
> >> + throw new RuntimeException( "Failed to write data", ex );
> >> + }
> >> }
> >>
> >> public void read( BufferedReader reader, ImportParams params )
> >>
> >> === modified file
> >>
> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/exporter/DefaultDhis14XMLExportService.java'
> >> ---
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/exporter/DefaultDhis14XMLExportService.java
> >> 2009-11-07 14:09:00 +0000
> >> +++
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/exporter/DefaultDhis14XMLExportService.java
> >> 2010-02-08 10:56:08 +0000
> >> @@ -35,16 +35,19 @@
> >> import java.util.zip.ZipEntry;
> >> import java.util.zip.ZipOutputStream;
> >>
> >> +import org.amplecode.quick.StatementManager;
> >> import org.amplecode.staxwax.factory.XMLFactory;
> >> import org.amplecode.staxwax.writer.XMLWriter;
> >> import org.hibernate.SessionFactory;
> >> import org.hisp.dhis.dataelement.DataElementService;
> >> +import org.hisp.dhis.datamart.DataMartService;
> >> import org.hisp.dhis.importexport.ExportParams;
> >> import org.hisp.dhis.importexport.ExportPipeThread;
> >> import org.hisp.dhis.importexport.ExportService;
> >> import
> >>
> org.hisp.dhis.importexport.dhis14.xml.converter.CalculatedDataElementAssociationConverter;
> >> import
> >> org.hisp.dhis.importexport.dhis14.xml.converter.DataElementConverter;
> >> import
> org.hisp.dhis.importexport.dhis14.xml.converter.DataTypeConverter;
> >> +import
> >> org.hisp.dhis.importexport.dhis14.xml.converter.DataValueConverter;
> >> import
> >> org.hisp.dhis.importexport.dhis14.xml.converter.IndicatorConverter;
> >> import
> >> org.hisp.dhis.importexport.dhis14.xml.converter.IndicatorTypeConverter;
> >> import
> >> org.hisp.dhis.importexport.dhis14.xml.converter.PeriodTypeConverter;
> >> @@ -60,6 +63,7 @@
> >> import
> >>
> org.hisp.dhis.importexport.dhis14.xml.converter.xsd.UserRoleXSDConverter;
> >> import
> >> org.hisp.dhis.importexport.dhis14.xml.converter.xsd.UserXSDConverter;
> >> import org.hisp.dhis.indicator.IndicatorService;
> >> +import org.hisp.dhis.period.PeriodService;
> >>
> >> /**
> >> * @author Lars Helge Overland
> >> @@ -100,6 +104,27 @@
> >> this.indicatorService = indicatorService;
> >> }
> >>
> >> + private PeriodService periodService;
> >> +
> >> + public void setPeriodService( PeriodService periodService )
> >> + {
> >> + this.periodService = periodService;
> >> + }
> >> +
> >> + private DataMartService dataMartService;
> >> +
> >> + public void setDataMartService( DataMartService dataMartService )
> >> + {
> >> + this.dataMartService = dataMartService;
> >> + }
> >> +
> >> + private StatementManager statementManager;
> >> +
> >> + public void setStatementManager( StatementManager statementManager
> )
> >> + {
> >> + this.statementManager = statementManager;
> >> + }
> >> +
> >> //
> >>
> -------------------------------------------------------------------------
> >> // ExportService implementation
> >> //
> >>
> -------------------------------------------------------------------------
> >> @@ -122,7 +147,7 @@
> >> zipOut.putNextEntry( new ZipEntry( "Export.xml" ) );
> >>
> >> XMLWriter writer = XMLFactory.getPlainXMLWriter( zipOut );
> >> -
> >> +
> >> //
> >>
> -------------------------------------------------------------------------
> >> // Writes to one end of the pipe
> >> //
> >>
> -------------------------------------------------------------------------
> >> @@ -156,6 +181,8 @@
> >> thread.registerXMLConverter( new UserConverter() );
> >> thread.registerXMLConverter( new UserRoleConverter() );
> >>
> >> + thread.registerCSVConverter( new DataValueConverter(
> >> periodService, dataMartService, statementManager ) );
> >> +
> >> thread.start();
> >>
> >> //
> >>
> -------------------------------------------------------------------------
> >>
> >> === modified file
> >>
> 'dhis-2/dhis-services/dhis-service-importexport/src/main/resources/META-INF/dhis/beans.xml'
> >> ---
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/resources/META-INF/dhis/beans.xml
> >> 2010-02-02 19:21:58 +0000
> >> +++
> >>
> dhis-2/dhis-services/dhis-service-importexport/src/main/resources/META-INF/dhis/beans.xml
> >> 2010-02-08 10:56:08 +0000
> >> @@ -381,6 +381,9 @@
> >> <property name="sessionFactory" ref="sessionFactory" />
> >> <property name="dataElementService"
> >> ref="org.hisp.dhis.dataelement.DataElementService" />
> >> <property name="indicatorService"
> >> ref="org.hisp.dhis.indicator.IndicatorService" />
> >> + <property name="periodService"
> >> ref="org.hisp.dhis.period.PeriodService" />
> >> + <property name="dataMartService"
> >> ref="org.hisp.dhis.datamart.DataMartService" />
> >> + <property name="statementManager" ref="statementManager"
> >> />
> >> </bean>
> >>
> >> <!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> -
> >> -->
> >>
> >> === modified file
> >>
> 'dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/DateUtils.java'
> >> ---
> >>
> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/DateUtils.java
> >> 2009-11-19 19:16:46 +0000
> >> +++
> >>
> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/DateUtils.java
> >> 2010-02-08 10:56:08 +0000
> >> @@ -49,6 +49,20 @@
> >> public static final String DEFAULT_DATE_FORMAT = "yyyy-MM-dd";
> >>
> >> /**
> >> + * Formats a Date to the Access date format.
> >> + *
> >> + * @param date the Date to parse.
> >> + * @return a formatted date string.
> >> + */
> >> + public static String getAccessDateString( Date date )
> >> + {
> >> + final SimpleDateFormat format = new SimpleDateFormat();
> >> + format.applyPattern( "yyyy/MM/dd HH:mm:ss" );
> >> +
> >> + return date != null ? format.format( date ) : null;
> >> + }
> >> +
> >> + /**
> >> * Formats a Date to the IXF date format which is
> >> YYYY-MM-DD'T'HH:MM:SS.
> >> *
> >> * @param date the Date to parse.
> >>
> >> === modified file
> >>
> 'dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/StreamUtils.java'
> >> ---
> >>
> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/StreamUtils.java
> >> 2009-12-19 15:20:41 +0000
> >> +++
> >>
> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/StreamUtils.java
> >> 2010-02-08 10:56:08 +0000
> >> @@ -422,9 +422,26 @@
> >> }
> >>
> >> /**
> >> + * Closes the current ZipEntry and positions the stream for writing
> >> the next entry.
> >> + *
> >> + * @param out the ZipOutputStream.
> >> + */
> >> + public static void closeZipEntry( ZipOutputStream out )
> >> + {
> >> + try
> >> + {
> >> + out.closeEntry();
> >> + }
> >> + catch ( Exception ex )
> >> + {
> >> + throw new RuntimeException( "Failed to close the current
> >> ZipEntry", ex );
> >> + }
> >> + }
> >> +
> >> + /**
> >> * Finishes writing the contents of the ZIP output stream without
> >> closing the underlying stream.
> >> *
> >> - * @param out the ZipOutputStream to write to.
> >> + * @param out the ZipOutputStream.
> >> */
> >> public static void finishZipEntry( ZipOutputStream out )
> >> {
> >> @@ -434,7 +451,7 @@
> >> }
> >> catch ( Exception ex )
> >> {
> >> - throw new RuntimeException( "Failed to finish
> >> ZipOutputStream", ex );
> >> + throw new RuntimeException( "Failed to finish the content
> of
> >> the ZipOutputStream", ex );
> >> }
> >> }
> >>
> >>
> >> === modified file
> >>
> 'dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties'
> >> ---
> >>
> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties
> >> 2010-01-27 21:45:11 +0000
> >> +++
> >>
> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties
> >> 2010-02-08 10:56:08 +0000
> >> @@ -146,6 +146,7 @@
> >> import_from_other_systems = Import from other systems
> >> DHIS14_import = DHIS 1.4 Import
> >> DHIS14_metadata_export = DHIS 1.4 Metadata Export
> >> +DHIS14_data_export = DHIS 1.4 Data Export
> >> IXF_import = IXF Import
> >> accept_incoming_records = Accept incoming records
> >> include_datavalues = Include data values
> >> @@ -442,4 +443,5 @@
> >> intro_ixf_metadata_export = Do an export of meta-data or dimensional
> data
> >> describing the facts. Indicator Transmission Format (IXF) is a standard
> >> developed by the WHO.
> >> intro_DHIS14_metadata_export = Do an export of meta-data or dimensional
> >> data describing the facts. DHIS 1.4 is the predecessor of DHIS 2.
> >> intro_DHIS14_detailed_metadata_export = Do an export of an detailed
> >> selection of meta-data. DHIS 1.4 is the predecessor of DHIS 2.
> >> +intro_DHIS14_data_export = Do an export of data values or facts. DHIS
> 1.4
> >> is the predecessor of DHIS 2.
> >> intro_pdf_metadata_export = Portable Document Format (PDF) is a file
> >> format for document exchange.
> >>
> >> === modified file
> >>
> 'dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/externalExportMenu.vm'
> >> ---
> >>
> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/externalExportMenu.vm
> >> 2010-01-28 11:51:39 +0000
> >> +++
> >>
> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/externalExportMenu.vm
> >> 2010-02-08 10:56:08 +0000
> >> @@ -2,8 +2,9 @@
> >> <h3>$i18n.getString( "export_to_other_systems" )</h3>
> >>
> >> <ul class="introList">
> >> + #introListItem( "displayMetaDataExportForm.action?exportFormat=IXF"
> >> "ixf_metadata_export" )
> >> #introListItem( "displayDataValueExportForm.action?exportFormat=IXF"
> >> "ixf_data_export" )
> >> - #introListItem( "displayMetaDataExportForm.action?exportFormat=IXF"
> >> "ixf_metadata_export" )
> >> #introListItem(
> >> "displayMetaDataExportForm.action?exportFormat=DHIS14XML"
> >> "DHIS14_metadata_export" )
> >> #introListItem(
> >> "displayDetailedMetaDataExportForm.action?exportFormat=DHIS14XML"
> >> "DHIS14_detailed_metadata_export" )
> >> + #introListItem(
> >> "displayDataValueExportForm.action?exportFormat=DHIS14XML"
> >> "DHIS14_data_export" )
> >> #introListItem( "displayMetaDataExportForm.action?exportFormat=PDF"
> >> "pdf_metadata_export" )
> >> \ No newline at end of file
> >>
> >>
> >> _______________________________________________
> >> Mailing list: https://launchpad.net/~dhis2-devs
> >> Post to : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >> Unsubscribe : https://launchpad.net/~dhis2-devs
> >> More help : https://help.launchpad.net/ListHelp
> >>
> >
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~dhis2-devs
> > Post to : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~dhis2-devs
> > More help : https://help.launchpad.net/ListHelp
> >
> >
>
Follow ups
References
