dhis2-devs team mailing list archive
-
dhis2-devs team
-
Mailing list archive
-
Message #04264
Re: [Branch ~dhis2-devs-core/dhis2/trunk] Rev 1404: Implemented DHIS 1.4 export of data values
Yup agree with that just could not make it write the appropriate zip
entries that way.. Open for being enlighted here..
2010/2/8, Bob Jolliffe <bobjolliffe@xxxxxxxxx>:
> 2010/2/8 Lars Helge Øverland <larshelge@xxxxxxxxx>:
>>
>>
>> 2010/2/8 Bob Jolliffe <bobjolliffe@xxxxxxxxx>
>>>
>>> 2010/2/8 Lars Helge Øverland <larshelge@xxxxxxxxx>:
>>> >
>>> >
>>> > On Mon, Feb 8, 2010 at 11:59 AM, <noreply@xxxxxxxxxxxxx> wrote:
>>> >>
>>> >> ------------------------------------------------------------
>>> >> revno: 1404
>>> >> committer: Lars Helge Oeverland <larshelge@xxxxxxxxx>
>>> >> branch nick: trunk
>>> >> timestamp: Mon 2010-02-08 11:56:08 +0100
>>> >> message:
>>> >> Implemented DHIS 1.4 export of data values
>>> >> modified:
>>> >
>>> >
>>> > There is actually still a snag here. DHIS 1.4 only accepts the
>>> > 7zip/lzma
>>> > format currently. I have talked Greg into making 1.4 accept zip/deflate
>>> > too.
>>>
>>> Isn't it better to have gzip/deflate? We are not talking about
>>> compressing an archive of files but rather compressing a single
>>> stream.
>>>
>>> Also I think the original "void write( BufferedWriter writer,
>>> ExportParams params )" might be better than "void write(
>>> ZipOutputStream out, ExportParams params )". It seems unnecessary
>>> that the convertor should have to know anything about zip, gzip, 7zip
>>> or what have you. It should know how to write csv to a stream.
>>>
>>> I suspect that how the stream is later compressed is better deferred to
>>> later.
>>
>> Problem is that DHIS 1.4 uses a CSV file for data and an XML file for
>> meta-data and keeps both inside the archive... I was was not able to have
>> multiple zip entries while using the Writer..
>
> Ah. Ok. I do remember. Then you would need an archive (zip or the
> seven thing) rather a gzipped stream.
>
> But does your CSV writer need to know that? Surely it just has an
> interest in churning out csv datavalues. Some other component should
> put the csv stream together with the xml stream into the zip.
>
>>
>>>
>>> Cheers
>>> Bob
>>>
>>> >
>>> >>
>>> >>
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/CSVConverter.java
>>> >>
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/ExportPipeThread.java
>>> >>
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/converter/ReportTableDataConverter.java
>>> >>
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/CSVExportPipeThread.java
>>> >>
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/DefaultCSVExportService.java
>>> >>
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/util/CsvUtil.java
>>> >>
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/converter/DataValueConverter.java
>>> >>
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/exporter/DefaultDhis14XMLExportService.java
>>> >>
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/resources/META-INF/dhis/beans.xml
>>> >>
>>> >>
>>> >> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/DateUtils.java
>>> >>
>>> >>
>>> >> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/StreamUtils.java
>>> >>
>>> >>
>>> >> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties
>>> >>
>>> >>
>>> >> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/externalExportMenu.vm
>>> >>
>>> >>
>>> >> --
>>> >> lp:dhis2
>>> >> https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk
>>> >>
>>> >> Your team DHIS 2 developers is subscribed to branch lp:dhis2.
>>> >> To unsubscribe from this branch go to
>>> >>
>>> >> https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription.
>>> >>
>>> >> === modified file
>>> >>
>>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/CSVConverter.java'
>>> >> ---
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/CSVConverter.java
>>> >> 2009-03-03 16:46:36 +0000
>>> >> +++
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/CSVConverter.java
>>> >> 2010-02-08 10:56:08 +0000
>>> >> @@ -28,7 +28,7 @@
>>> >> */
>>> >>
>>> >> import java.io.BufferedReader;
>>> >> -import java.io.BufferedWriter;
>>> >> +import java.util.zip.ZipOutputStream;
>>> >>
>>> >> /**
>>> >> * @author Lars Helge Overland
>>> >> @@ -36,7 +36,7 @@
>>> >> */
>>> >> public interface CSVConverter
>>> >> {
>>> >> - void write( BufferedWriter writer, ExportParams params );
>>> >> + void write( ZipOutputStream out, ExportParams params );
>>> >>
>>> >> void read( BufferedReader reader, ImportParams params );
>>> >> }
>>> >>
>>> >> === modified file
>>> >>
>>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/ExportPipeThread.java'
>>> >> ---
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/ExportPipeThread.java
>>> >> 2009-11-02 15:55:44 +0000
>>> >> +++
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/ExportPipeThread.java
>>> >> 2010-02-08 10:56:08 +0000
>>> >> @@ -52,7 +52,8 @@
>>> >>
>>> >> private List<XMLConverter> xsdConverters = new
>>> >> ArrayList<XMLConverter>();
>>> >> private List<XMLConverter> xmlConverters = new
>>> >> ArrayList<XMLConverter>();
>>> >> -
>>> >> + private List<CSVConverter> csvConverters = new
>>> >> ArrayList<CSVConverter>();
>>> >> +
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >> // Parameters
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >> @@ -122,6 +123,11 @@
>>> >> this.xmlConverters.add( converter );
>>> >> }
>>> >>
>>> >> + public void registerCSVConverter ( CSVConverter converter )
>>> >> + {
>>> >> + this.csvConverters.add( converter );
>>> >> + }
>>> >> +
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >> // Thread implementation
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >> @@ -159,9 +165,20 @@
>>> >> }
>>> >>
>>> >> afterXML( writer );
>>> >> -
>>> >> +
>>> >> closeDocument( writer );
>>> >> -
>>> >> +
>>> >> + StreamUtils.closeZipEntry( zipOutputStream );
>>> >> +
>>> >> + //
>>> >> -----------------------------------------------------------------
>>> >> + // CSV
>>> >> + //
>>> >> -----------------------------------------------------------------
>>> >> +
>>> >> + for ( CSVConverter converter : csvConverters )
>>> >> + {
>>> >> + converter.write( zipOutputStream, params );
>>> >> + }
>>> >> +
>>> >> log.info( "Export done" );
>>> >> }
>>> >> catch ( Exception ex )
>>> >> @@ -172,12 +189,10 @@
>>> >> }
>>> >> finally
>>> >> {
>>> >> - StreamUtils.finishZipEntry( zipOutputStream );
>>> >> + writer.closeWriter();
>>> >>
>>> >> StreamUtils.closeOutputStream( zipOutputStream );
>>> >>
>>> >> - writer.closeWriter();
>>> >> -
>>> >> NameMappingUtil.clearMapping();
>>> >> }
>>> >> }
>>> >>
>>> >> === modified file
>>> >>
>>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/converter/ReportTableDataConverter.java'
>>> >> ---
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/converter/ReportTableDataConverter.java
>>> >> 2009-06-10 22:25:07 +0000
>>> >> +++
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/converter/ReportTableDataConverter.java
>>> >> 2010-02-08 10:56:08 +0000
>>> >> @@ -27,14 +27,17 @@
>>> >> * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>>> >> */
>>> >>
>>> >> -import static org.hisp.dhis.importexport.csv.util.CsvUtil.SEPARATOR;
>>> >> +import static
>>> >> org.hisp.dhis.importexport.csv.util.CsvUtil.CSV_EXTENSION;
>>> >> +import static org.hisp.dhis.importexport.csv.util.CsvUtil.NEWLINE;
>>> >> +import static
>>> >> org.hisp.dhis.importexport.csv.util.CsvUtil.SEPARATOR_B;
>>> >> import static org.hisp.dhis.importexport.csv.util.CsvUtil.csvEncode;
>>> >>
>>> >> import java.io.BufferedReader;
>>> >> -import java.io.BufferedWriter;
>>> >> import java.io.IOException;
>>> >> import java.util.Iterator;
>>> >> import java.util.SortedMap;
>>> >> +import java.util.zip.ZipEntry;
>>> >> +import java.util.zip.ZipOutputStream;
>>> >>
>>> >> import org.hisp.dhis.importexport.CSVConverter;
>>> >> import org.hisp.dhis.importexport.ExportParams;
>>> >> @@ -69,27 +72,29 @@
>>> >> // CSVConverter implementation
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >>
>>> >> - public void write( BufferedWriter writer, ExportParams params )
>>> >> + public void write( ZipOutputStream out, ExportParams params )
>>> >> {
>>> >> try
>>> >> {
>>> >> - for ( Integer id : params.getReportTables() ) //TODO more
>>> >> than one?
>>> >> + for ( Integer id : params.getReportTables() )
>>> >> {
>>> >> + out.putNextEntry( new ZipEntry( "ReportTable" + id +
>>> >> CSV_EXTENSION ) );
>>> >> +
>>> >> ReportTableData data =
>>> >> reportTableService.getReportTableData( id, params.getFormat() );
>>> >>
>>> >> Iterator<String> columns =
>>> >> data.getPrettyPrintColumns().iterator();
>>> >>
>>> >> while ( columns.hasNext() )
>>> >> {
>>> >> - writer.write( csvEncode( columns.next() ) );
>>> >> + out.write( csvEncode( columns.next() ).getBytes()
>>> >> );
>>> >>
>>> >> if ( columns.hasNext() )
>>> >> {
>>> >> - writer.write( SEPARATOR );
>>> >> + out.write( SEPARATOR_B );
>>> >> }
>>> >> }
>>> >>
>>> >> - writer.newLine();
>>> >> + out.write( NEWLINE );
>>> >>
>>> >> for ( SortedMap<Integer, String> row : data.getRows()
>>> >> )
>>> >> {
>>> >> @@ -97,15 +102,15 @@
>>> >>
>>> >> while ( values.hasNext() )
>>> >> {
>>> >> - writer.write( csvEncode( values.next() ) );
>>> >> + out.write( csvEncode( values.next()
>>> >> ).getBytes()
>>> >> );
>>> >>
>>> >> if ( values.hasNext() )
>>> >> {
>>> >> - writer.write( SEPARATOR );
>>> >> + out.write( SEPARATOR_B );
>>> >> }
>>> >> }
>>> >>
>>> >> - writer.newLine();
>>> >> + out.write( NEWLINE );
>>> >> }
>>> >> }
>>> >> }
>>> >>
>>> >> === modified file
>>> >>
>>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/CSVExportPipeThread.java'
>>> >> ---
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/CSVExportPipeThread.java
>>> >> 2009-06-10 22:25:07 +0000
>>> >> +++
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/CSVExportPipeThread.java
>>> >> 2010-02-08 10:56:08 +0000
>>> >> @@ -27,9 +27,9 @@
>>> >> * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>>> >> */
>>> >>
>>> >> -import java.io.BufferedWriter;
>>> >> import java.util.ArrayList;
>>> >> import java.util.List;
>>> >> +import java.util.zip.ZipOutputStream;
>>> >>
>>> >> import org.apache.commons.logging.Log;
>>> >> import org.apache.commons.logging.LogFactory;
>>> >> @@ -48,13 +48,6 @@
>>> >> {
>>> >> private static final Log log = LogFactory.getLog(
>>> >> CSVExportPipeThread.class );
>>> >>
>>> >> - private BufferedWriter writer;
>>> >> -
>>> >> - public void setWriter( BufferedWriter writer )
>>> >> - {
>>> >> - this.writer = writer;
>>> >> - }
>>> >> -
>>> >> private ExportParams params;
>>> >>
>>> >> public void setParams( ExportParams params )
>>> >> @@ -62,6 +55,13 @@
>>> >> this.params = params;
>>> >> }
>>> >>
>>> >> + private ZipOutputStream outputStream;
>>> >> +
>>> >> + public void setOutputStream( ZipOutputStream outputStream )
>>> >> + {
>>> >> + this.outputStream = outputStream;
>>> >> + }
>>> >> +
>>> >> private List<CSVConverter> converters = new
>>> >> ArrayList<CSVConverter>();
>>> >>
>>> >> public void registerCSVConverter( CSVConverter converter )
>>> >> @@ -90,14 +90,18 @@
>>> >>
>>> >> for ( CSVConverter converter : converters )
>>> >> {
>>> >> - converter.write( writer, params );
>>> >> + converter.write( outputStream, params );
>>> >> }
>>> >>
>>> >> log.info( "Export finished" );
>>> >> }
>>> >> + catch ( Exception ex )
>>> >> + {
>>> >> + throw new RuntimeException( ex );
>>> >> + }
>>> >> finally
>>> >> {
>>> >> - StreamUtils.closeWriter( writer );
>>> >> + StreamUtils.closeOutputStream( outputStream );
>>> >> }
>>> >> }
>>> >> }
>>> >>
>>> >> === modified file
>>> >>
>>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/DefaultCSVExportService.java'
>>> >> ---
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/DefaultCSVExportService.java
>>> >> 2009-06-10 22:25:07 +0000
>>> >> +++
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/DefaultCSVExportService.java
>>> >> 2010-02-08 10:56:08 +0000
>>> >> @@ -28,13 +28,11 @@
>>> >> */
>>> >>
>>> >> import java.io.BufferedInputStream;
>>> >> -import java.io.BufferedWriter;
>>> >> +import java.io.BufferedOutputStream;
>>> >> import java.io.IOException;
>>> >> import java.io.InputStream;
>>> >> -import java.io.OutputStreamWriter;
>>> >> import java.io.PipedInputStream;
>>> >> import java.io.PipedOutputStream;
>>> >> -import java.util.zip.ZipEntry;
>>> >> import java.util.zip.ZipOutputStream;
>>> >>
>>> >> import org.hibernate.SessionFactory;
>>> >> @@ -50,8 +48,6 @@
>>> >> public class DefaultCSVExportService
>>> >> implements ExportService
>>> >> {
>>> >> - private static final String ZIP_ENTRY_NAME = "Export.csv";
>>> >> -
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >> // Dependencies
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >> @@ -87,20 +83,18 @@
>>> >>
>>> >> PipedInputStream in = new PipedInputStream( out );
>>> >>
>>> >> - ZipOutputStream zipOut = new ZipOutputStream( out );
>>> >> + ZipOutputStream zipOut = new ZipOutputStream( new
>>> >> BufferedOutputStream( out ) );
>>> >>
>>> >> - zipOut.putNextEntry( new ZipEntry( ZIP_ENTRY_NAME ) );
>>> >> + //zipOut.putNextEntry( new ZipEntry( ZIP_ENTRY_NAME ) );
>>> >>
>>> >> - BufferedWriter writer = new BufferedWriter( new
>>> >> OutputStreamWriter( zipOut ) );
>>> >> -
>>> >> //
>>> >> -----------------------------------------------------------------
>>> >> // Writes to one end of the pipe
>>> >> //
>>> >> -----------------------------------------------------------------
>>> >>
>>> >> CSVExportPipeThread thread = new CSVExportPipeThread(
>>> >> sessionFactory );
>>> >>
>>> >> - thread.setWriter( writer );
>>> >> thread.setParams( params );
>>> >> + thread.setOutputStream( zipOut );
>>> >>
>>> >> thread.registerCSVConverter( new ReportTableDataConverter(
>>> >> reportTableService ) );
>>> >>
>>> >>
>>> >> === modified file
>>> >>
>>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/util/CsvUtil.java'
>>> >> ---
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/util/CsvUtil.java
>>> >> 2009-04-16 09:53:17 +0000
>>> >> +++
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/util/CsvUtil.java
>>> >> 2010-02-08 10:56:08 +0000
>>> >> @@ -33,15 +33,80 @@
>>> >> */
>>> >> public class CsvUtil
>>> >> {
>>> >> - public static final char SEPARATOR = ',';
>>> >> -
>>> >> - private static final String ENCLOSURE = "\"";
>>> >> -
>>> >> - public static String csvEncode( String string )
>>> >> - {
>>> >> - string = string.replaceAll( ENCLOSURE, ENCLOSURE + ENCLOSURE
>>> >> );
>>> >> - string = ENCLOSURE + string + ENCLOSURE;
>>> >> + public static final String SEPARATOR = ",";
>>> >> + public static final byte[] SEPARATOR_B = SEPARATOR.getBytes();
>>> >> + public static final byte[] NEWLINE = "\n".getBytes();
>>> >> +
>>> >> + public static final String CSV_EXTENSION = ".csv";
>>> >> + private static final String ENCLOSURE = "\"";
>>> >> + private static final String EMPTY = "";
>>> >> +
>>> >> + /**
>>> >> + * Encodes the given value to a CSV acceptable value.
>>> >> + *
>>> >> + * @param value the value.
>>> >> + * @return the CSV encoded value.
>>> >> + */
>>> >> + public static String csvEncode( int value )
>>> >> + {
>>> >> + return csvEncode( String.valueOf( value ) );
>>> >> + }
>>> >> +
>>> >> + /**
>>> >> + * Encodes the given value to a CSV acceptable value.
>>> >> + *
>>> >> + * @param value the value.
>>> >> + * @return the CSV encoded value.
>>> >> + */
>>> >> + public static String csvEncode( String value )
>>> >> + {
>>> >> + if ( value == null )
>>> >> + {
>>> >> + value = EMPTY;
>>> >> + }
>>> >> + else
>>> >> + {
>>> >> + value = value.replaceAll( ENCLOSURE, ENCLOSURE +
>>> >> ENCLOSURE
>>> >> );
>>> >> + value = ENCLOSURE + value + ENCLOSURE;
>>> >> + }
>>> >> +
>>> >> + return value;
>>> >> + }
>>> >> +
>>> >> + /**
>>> >> + * Appends a separator to the value and returns the value as a
>>> >> byte
>>> >> array.
>>> >> + *
>>> >> + * @param value the value.
>>> >> + * @return a byte araray.
>>> >> + */
>>> >> + public static byte[] getCsvValue( int value )
>>> >> + {
>>> >> + return getCsvEndValue( value + SEPARATOR );
>>> >> + }
>>> >> +
>>> >> + /**
>>> >> + * Appends a separator to the value and returns the value as a
>>> >> byte
>>> >> array.
>>> >> + *
>>> >> + * @param value the value.
>>> >> + * @return a byte araray.
>>> >> + */
>>> >> + public static byte[] getCsvValue( String value )
>>> >> + {
>>> >> + return getCsvEndValue( value + SEPARATOR );
>>> >> + }
>>> >> +
>>> >> + public static byte[] getCsvEndValue( int value )
>>> >> + {
>>> >> + return getCsvEndValue( String.valueOf( value ) );
>>> >> + }
>>> >> +
>>> >> + public static byte[] getCsvEndValue( String value )
>>> >> + {
>>> >> + if ( value == null )
>>> >> + {
>>> >> + return EMPTY.getBytes();
>>> >> + }
>>> >>
>>> >> - return string;
>>> >> + return ( value ).getBytes();
>>> >> }
>>> >> }
>>> >>
>>> >> === modified file
>>> >>
>>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/converter/DataValueConverter.java'
>>> >> ---
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/converter/DataValueConverter.java
>>> >> 2010-02-08 07:04:26 +0000
>>> >> +++
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/converter/DataValueConverter.java
>>> >> 2010-02-08 10:56:08 +0000
>>> >> @@ -27,17 +27,28 @@
>>> >> * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>>> >> */
>>> >>
>>> >> +import static org.hisp.dhis.importexport.csv.util.CsvUtil.NEWLINE;
>>> >> +import static
>>> >> org.hisp.dhis.importexport.csv.util.CsvUtil.SEPARATOR_B;
>>> >> +import static org.hisp.dhis.importexport.csv.util.CsvUtil.csvEncode;
>>> >> +import static
>>> >> org.hisp.dhis.importexport.csv.util.CsvUtil.getCsvValue;
>>> >> +import static
>>> >> org.hisp.dhis.importexport.csv.util.CsvUtil.getCsvEndValue;
>>> >> +
>>> >> import java.io.BufferedReader;
>>> >> -import java.io.BufferedWriter;
>>> >> import java.io.IOException;
>>> >> +import java.util.Collection;
>>> >> import java.util.Map;
>>> >> +import java.util.zip.ZipEntry;
>>> >> +import java.util.zip.ZipOutputStream;
>>> >>
>>> >> import org.amplecode.quick.BatchHandler;
>>> >> +import org.amplecode.quick.StatementManager;
>>> >> import org.hisp.dhis.dataelement.DataElement;
>>> >> import org.hisp.dhis.dataelement.DataElementCategoryOptionCombo;
>>> >> import org.hisp.dhis.dataelement.DataElementCategoryService;
>>> >> +import org.hisp.dhis.datamart.DataMartService;
>>> >> import org.hisp.dhis.datavalue.DataValue;
>>> >> import org.hisp.dhis.datavalue.DataValueService;
>>> >> +import org.hisp.dhis.datavalue.DeflatedDataValue;
>>> >> import org.hisp.dhis.importexport.CSVConverter;
>>> >> import org.hisp.dhis.importexport.ExportParams;
>>> >> import org.hisp.dhis.importexport.GroupMemberType;
>>> >> @@ -47,7 +58,10 @@
>>> >> import
>>> >> org.hisp.dhis.importexport.converter.AbstractDataValueConverter;
>>> >> import org.hisp.dhis.organisationunit.OrganisationUnit;
>>> >> import org.hisp.dhis.period.Period;
>>> >> +import org.hisp.dhis.period.PeriodService;
>>> >> +import org.hisp.dhis.system.util.DateUtils;
>>> >> import org.hisp.dhis.system.util.MimicingHashMap;
>>> >> +import org.hisp.dhis.system.util.StreamUtils;
>>> >>
>>> >> /**
>>> >> * @author Lars Helge Overland
>>> >> @@ -57,8 +71,11 @@
>>> >> extends AbstractDataValueConverter implements CSVConverter
>>> >> {
>>> >> private static final String SEPARATOR = ",";
>>> >> + private static final String FILENAME = "RoutineData.txt";
>>> >>
>>> >> private DataElementCategoryService categoryService;
>>> >> + private PeriodService periodService;
>>> >> + private StatementManager statementManager;
>>> >>
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >> // Properties
>>> >> @@ -72,6 +89,14 @@
>>> >> // Constructor
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >>
>>> >> + public DataValueConverter( PeriodService periodService,
>>> >> DataMartService dataMartService,
>>> >> + StatementManager statementManager )
>>> >> + {
>>> >> + this.periodService = periodService;
>>> >> + this.dataMartService = dataMartService;
>>> >> + this.statementManager = statementManager;
>>> >> + }
>>> >> +
>>> >> /**
>>> >> * Constructor for read operations.
>>> >> */
>>> >> @@ -95,9 +120,81 @@
>>> >> // CSVConverter implementation
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >>
>>> >> - public void write( BufferedWriter writer, ExportParams params )
>>> >> + public void write( ZipOutputStream out, ExportParams params )
>>> >> {
>>> >> - // Not implemented
>>> >> + try
>>> >> + {
>>> >> + out.putNextEntry( new ZipEntry( FILENAME ) );
>>> >> +
>>> >> + out.write( getCsvValue( csvEncode( "RoutineDataID" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "OrgUnitID" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "DataElementID" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "DataPeriodID" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "EntryText" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "EntryYesNo" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "EntryNumber" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "EntryDate" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "EntryMemo" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "EntryObject" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "Check" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "Verified" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "Deleted" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "Comment" ) ) );
>>> >> + out.write( getCsvValue( csvEncode( "LastUserID" ) ) );
>>> >> + out.write( getCsvEndValue( csvEncode( "LastUpdated" ) )
>>> >> );
>>> >> +
>>> >> + out.write( NEWLINE );
>>> >> +
>>> >> + if ( params.isIncludeDataValues() )
>>> >> + {
>>> >> + if ( params.getStartDate() != null &&
>>> >> params.getEndDate()
>>> >> != null )
>>> >> + {
>>> >> + Collection<DeflatedDataValue> values = null;
>>> >> +
>>> >> + Collection<Period> periods =
>>> >> periodService.getIntersectingPeriods( params.getStartDate(),
>>> >> params.getEndDate() );
>>> >> +
>>> >> + statementManager.initialise();
>>> >> +
>>> >> + for ( final Integer element :
>>> >> params.getDataElements() )
>>> >> + {
>>> >> + for ( final Period period : periods )
>>> >> + {
>>> >> + values =
>>> >> dataMartService.getDeflatedDataValues( element, period.getId(),
>>> >> params.getOrganisationUnits() );
>>> >> +
>>> >> + for ( final DeflatedDataValue value :
>>> >> values
>>> >> )
>>> >> + {
>>> >> + out.write( getCsvValue( 0 ) );
>>> >> + out.write( getCsvValue(
>>> >> value.getSourceId() ) );
>>> >> + out.write( getCsvValue(
>>> >> value.getDataElementId() ) );
>>> >> + out.write( getCsvValue(
>>> >> value.getPeriodId() ) );
>>> >> + out.write( SEPARATOR_B );
>>> >> + out.write( SEPARATOR_B );
>>> >> + out.write( getCsvValue( csvEncode(
>>> >> value.getValue() ) ) );
>>> >> + out.write( SEPARATOR_B );
>>> >> + out.write( SEPARATOR_B );
>>> >> + out.write( SEPARATOR_B );
>>> >> + out.write( getCsvValue( 0 ) );
>>> >> + out.write( getCsvValue( 0 ) );
>>> >> + out.write( getCsvValue( 0 ) );
>>> >> + out.write( getCsvValue( csvEncode(
>>> >> value.getComment() ) ) );
>>> >> + out.write( getCsvValue( 1 ) );
>>> >> + out.write( getCsvEndValue(
>>> >> DateUtils.getAccessDateString( value.getTimestamp() ) ) );
>>> >> +
>>> >> + out.write( NEWLINE );
>>> >> + }
>>> >> + }
>>> >> + }
>>> >> +
>>> >> + statementManager.destroy();
>>> >> + }
>>> >> + }
>>> >> +
>>> >> + StreamUtils.closeZipEntry( out );
>>> >> + }
>>> >> + catch ( IOException ex )
>>> >> + {
>>> >> + throw new RuntimeException( "Failed to write data", ex );
>>> >> + }
>>> >> }
>>> >>
>>> >> public void read( BufferedReader reader, ImportParams params )
>>> >>
>>> >> === modified file
>>> >>
>>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/exporter/DefaultDhis14XMLExportService.java'
>>> >> ---
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/exporter/DefaultDhis14XMLExportService.java
>>> >> 2009-11-07 14:09:00 +0000
>>> >> +++
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/exporter/DefaultDhis14XMLExportService.java
>>> >> 2010-02-08 10:56:08 +0000
>>> >> @@ -35,16 +35,19 @@
>>> >> import java.util.zip.ZipEntry;
>>> >> import java.util.zip.ZipOutputStream;
>>> >>
>>> >> +import org.amplecode.quick.StatementManager;
>>> >> import org.amplecode.staxwax.factory.XMLFactory;
>>> >> import org.amplecode.staxwax.writer.XMLWriter;
>>> >> import org.hibernate.SessionFactory;
>>> >> import org.hisp.dhis.dataelement.DataElementService;
>>> >> +import org.hisp.dhis.datamart.DataMartService;
>>> >> import org.hisp.dhis.importexport.ExportParams;
>>> >> import org.hisp.dhis.importexport.ExportPipeThread;
>>> >> import org.hisp.dhis.importexport.ExportService;
>>> >> import
>>> >>
>>> >> org.hisp.dhis.importexport.dhis14.xml.converter.CalculatedDataElementAssociationConverter;
>>> >> import
>>> >> org.hisp.dhis.importexport.dhis14.xml.converter.DataElementConverter;
>>> >> import
>>> >> org.hisp.dhis.importexport.dhis14.xml.converter.DataTypeConverter;
>>> >> +import
>>> >> org.hisp.dhis.importexport.dhis14.xml.converter.DataValueConverter;
>>> >> import
>>> >> org.hisp.dhis.importexport.dhis14.xml.converter.IndicatorConverter;
>>> >> import
>>> >> org.hisp.dhis.importexport.dhis14.xml.converter.IndicatorTypeConverter;
>>> >> import
>>> >> org.hisp.dhis.importexport.dhis14.xml.converter.PeriodTypeConverter;
>>> >> @@ -60,6 +63,7 @@
>>> >> import
>>> >>
>>> >> org.hisp.dhis.importexport.dhis14.xml.converter.xsd.UserRoleXSDConverter;
>>> >> import
>>> >> org.hisp.dhis.importexport.dhis14.xml.converter.xsd.UserXSDConverter;
>>> >> import org.hisp.dhis.indicator.IndicatorService;
>>> >> +import org.hisp.dhis.period.PeriodService;
>>> >>
>>> >> /**
>>> >> * @author Lars Helge Overland
>>> >> @@ -100,6 +104,27 @@
>>> >> this.indicatorService = indicatorService;
>>> >> }
>>> >>
>>> >> + private PeriodService periodService;
>>> >> +
>>> >> + public void setPeriodService( PeriodService periodService )
>>> >> + {
>>> >> + this.periodService = periodService;
>>> >> + }
>>> >> +
>>> >> + private DataMartService dataMartService;
>>> >> +
>>> >> + public void setDataMartService( DataMartService dataMartService )
>>> >> + {
>>> >> + this.dataMartService = dataMartService;
>>> >> + }
>>> >> +
>>> >> + private StatementManager statementManager;
>>> >> +
>>> >> + public void setStatementManager( StatementManager
>>> >> statementManager
>>> >> )
>>> >> + {
>>> >> + this.statementManager = statementManager;
>>> >> + }
>>> >> +
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >> // ExportService implementation
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >> @@ -122,7 +147,7 @@
>>> >> zipOut.putNextEntry( new ZipEntry( "Export.xml" ) );
>>> >>
>>> >> XMLWriter writer = XMLFactory.getPlainXMLWriter( zipOut );
>>> >> -
>>> >> +
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >> // Writes to one end of the pipe
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >> @@ -156,6 +181,8 @@
>>> >> thread.registerXMLConverter( new UserConverter() );
>>> >> thread.registerXMLConverter( new UserRoleConverter() );
>>> >>
>>> >> + thread.registerCSVConverter( new DataValueConverter(
>>> >> periodService, dataMartService, statementManager ) );
>>> >> +
>>> >> thread.start();
>>> >>
>>> >> //
>>> >>
>>> >> -------------------------------------------------------------------------
>>> >>
>>> >> === modified file
>>> >>
>>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/resources/META-INF/dhis/beans.xml'
>>> >> ---
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/resources/META-INF/dhis/beans.xml
>>> >> 2010-02-02 19:21:58 +0000
>>> >> +++
>>> >>
>>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/resources/META-INF/dhis/beans.xml
>>> >> 2010-02-08 10:56:08 +0000
>>> >> @@ -381,6 +381,9 @@
>>> >> <property name="sessionFactory" ref="sessionFactory" />
>>> >> <property name="dataElementService"
>>> >> ref="org.hisp.dhis.dataelement.DataElementService" />
>>> >> <property name="indicatorService"
>>> >> ref="org.hisp.dhis.indicator.IndicatorService" />
>>> >> + <property name="periodService"
>>> >> ref="org.hisp.dhis.period.PeriodService" />
>>> >> + <property name="dataMartService"
>>> >> ref="org.hisp.dhis.datamart.DataMartService" />
>>> >> + <property name="statementManager"
>>> >> ref="statementManager"
>>> >> />
>>> >> </bean>
>>> >>
>>> >> <!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>> >> -
>>> >> -
>>> >> -->
>>> >>
>>> >> === modified file
>>> >>
>>> >> 'dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/DateUtils.java'
>>> >> ---
>>> >>
>>> >> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/DateUtils.java
>>> >> 2009-11-19 19:16:46 +0000
>>> >> +++
>>> >>
>>> >> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/DateUtils.java
>>> >> 2010-02-08 10:56:08 +0000
>>> >> @@ -49,6 +49,20 @@
>>> >> public static final String DEFAULT_DATE_FORMAT = "yyyy-MM-dd";
>>> >>
>>> >> /**
>>> >> + * Formats a Date to the Access date format.
>>> >> + *
>>> >> + * @param date the Date to parse.
>>> >> + * @return a formatted date string.
>>> >> + */
>>> >> + public static String getAccessDateString( Date date )
>>> >> + {
>>> >> + final SimpleDateFormat format = new SimpleDateFormat();
>>> >> + format.applyPattern( "yyyy/MM/dd HH:mm:ss" );
>>> >> +
>>> >> + return date != null ? format.format( date ) : null;
>>> >> + }
>>> >> +
>>> >> + /**
>>> >> * Formats a Date to the IXF date format which is
>>> >> YYYY-MM-DD'T'HH:MM:SS.
>>> >> *
>>> >> * @param date the Date to parse.
>>> >>
>>> >> === modified file
>>> >>
>>> >> 'dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/StreamUtils.java'
>>> >> ---
>>> >>
>>> >> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/StreamUtils.java
>>> >> 2009-12-19 15:20:41 +0000
>>> >> +++
>>> >>
>>> >> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/StreamUtils.java
>>> >> 2010-02-08 10:56:08 +0000
>>> >> @@ -422,9 +422,26 @@
>>> >> }
>>> >>
>>> >> /**
>>> >> + * Closes the current ZipEntry and positions the stream for
>>> >> writing
>>> >> the next entry.
>>> >> + *
>>> >> + * @param out the ZipOutputStream.
>>> >> + */
>>> >> + public static void closeZipEntry( ZipOutputStream out )
>>> >> + {
>>> >> + try
>>> >> + {
>>> >> + out.closeEntry();
>>> >> + }
>>> >> + catch ( Exception ex )
>>> >> + {
>>> >> + throw new RuntimeException( "Failed to close the current
>>> >> ZipEntry", ex );
>>> >> + }
>>> >> + }
>>> >> +
>>> >> + /**
>>> >> * Finishes writing the contents of the ZIP output stream without
>>> >> closing the underlying stream.
>>> >> *
>>> >> - * @param out the ZipOutputStream to write to.
>>> >> + * @param out the ZipOutputStream.
>>> >> */
>>> >> public static void finishZipEntry( ZipOutputStream out )
>>> >> {
>>> >> @@ -434,7 +451,7 @@
>>> >> }
>>> >> catch ( Exception ex )
>>> >> {
>>> >> - throw new RuntimeException( "Failed to finish
>>> >> ZipOutputStream", ex );
>>> >> + throw new RuntimeException( "Failed to finish the content
>>> >> of
>>> >> the ZipOutputStream", ex );
>>> >> }
>>> >> }
>>> >>
>>> >>
>>> >> === modified file
>>> >>
>>> >> 'dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties'
>>> >> ---
>>> >>
>>> >> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties
>>> >> 2010-01-27 21:45:11 +0000
>>> >> +++
>>> >>
>>> >> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties
>>> >> 2010-02-08 10:56:08 +0000
>>> >> @@ -146,6 +146,7 @@
>>> >> import_from_other_systems = Import from other systems
>>> >> DHIS14_import = DHIS 1.4 Import
>>> >> DHIS14_metadata_export = DHIS 1.4 Metadata Export
>>> >> +DHIS14_data_export = DHIS 1.4 Data Export
>>> >> IXF_import = IXF Import
>>> >> accept_incoming_records = Accept incoming records
>>> >> include_datavalues = Include data values
>>> >> @@ -442,4 +443,5 @@
>>> >> intro_ixf_metadata_export = Do an export of meta-data or dimensional
>>> >> data
>>> >> describing the facts. Indicator Transmission Format (IXF) is a
>>> >> standard
>>> >> developed by the WHO.
>>> >> intro_DHIS14_metadata_export = Do an export of meta-data or
>>> >> dimensional
>>> >> data describing the facts. DHIS 1.4 is the predecessor of DHIS 2.
>>> >> intro_DHIS14_detailed_metadata_export = Do an export of an detailed
>>> >> selection of meta-data. DHIS 1.4 is the predecessor of DHIS 2.
>>> >> +intro_DHIS14_data_export = Do an export of data values or facts. DHIS
>>> >> 1.4
>>> >> is the predecessor of DHIS 2.
>>> >> intro_pdf_metadata_export = Portable Document Format (PDF) is a file
>>> >> format for document exchange.
>>> >>
>>> >> === modified file
>>> >>
>>> >> 'dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/externalExportMenu.vm'
>>> >> ---
>>> >>
>>> >> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/externalExportMenu.vm
>>> >> 2010-01-28 11:51:39 +0000
>>> >> +++
>>> >>
>>> >> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/externalExportMenu.vm
>>> >> 2010-02-08 10:56:08 +0000
>>> >> @@ -2,8 +2,9 @@
>>> >> <h3>$i18n.getString( "export_to_other_systems" )</h3>
>>> >>
>>> >> <ul class="introList">
>>> >> + #introListItem(
>>> >> "displayMetaDataExportForm.action?exportFormat=IXF"
>>> >> "ixf_metadata_export" )
>>> >> #introListItem(
>>> >> "displayDataValueExportForm.action?exportFormat=IXF"
>>> >> "ixf_data_export" )
>>> >> - #introListItem(
>>> >> "displayMetaDataExportForm.action?exportFormat=IXF"
>>> >> "ixf_metadata_export" )
>>> >> #introListItem(
>>> >> "displayMetaDataExportForm.action?exportFormat=DHIS14XML"
>>> >> "DHIS14_metadata_export" )
>>> >> #introListItem(
>>> >> "displayDetailedMetaDataExportForm.action?exportFormat=DHIS14XML"
>>> >> "DHIS14_detailed_metadata_export" )
>>> >> + #introListItem(
>>> >> "displayDataValueExportForm.action?exportFormat=DHIS14XML"
>>> >> "DHIS14_data_export" )
>>> >> #introListItem(
>>> >> "displayMetaDataExportForm.action?exportFormat=PDF"
>>> >> "pdf_metadata_export" )
>>> >> \ No newline at end of file
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> Mailing list: https://launchpad.net/~dhis2-devs
>>> >> Post to : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> >> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> >> More help : https://help.launchpad.net/ListHelp
>>> >>
>>> >
>>> >
>>> > _______________________________________________
>>> > Mailing list: https://launchpad.net/~dhis2-devs
>>> > Post to : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> > Unsubscribe : https://launchpad.net/~dhis2-devs
>>> > More help : https://help.launchpad.net/ListHelp
>>> >
>>> >
>>
>>
>
--
Sendt fra min mobile enhet
Follow ups
References
