← Back to team overview

dhis2-devs team mailing list archive

Re: DPs Data Validated Data Inventory

 

Dear Devs

Please ignore this communication. It was not meant for this list.

Thanks.
Ismail

On Sat, 9 Jun 2018, 14:11 Ismail Yusuf, <ismailkoleleni@xxxxxxxxx> wrote:

> Mukulu,
>
> Please response back to Chris ASAP.
>
> ---------- Forwarded message ---------
> From: Chris Brown <chris.brown@xxxxxxxxxxxxxx>
> Date: Fri, Jun 8, 2018 at 5:36 PM
> Subject: Re: DPs Data Validated Data Inventory
> To: Ismail Yusuf <ismailkoleleni@xxxxxxxxx>
> Cc: M.M. Kissaka <mkissaka@xxxxxxxxx>, Chris Brown <
> chris.rm.brown@xxxxxxxxx>, tuzo engelbert <tuzoengelbert@xxxxxxxxx>, John
> mukulu <john.f.mukulu@xxxxxxxxx>, Hassan Omary <muhomis@xxxxxxxxx>,
> Angelina Misso <angelzm101@xxxxxxxxx>, Dr. Burchard Bagile <
> bbagile@xxxxxxxxx>
>
>
> Thank you Ismail.
>
> I still haven't heard from John.
>
> Can you please advise on whether we are going to hear back from CoICT this
> week as was agreed, so that we can make the necessary actions on the local
> government training that has been arranged by the Ministry of Water and
> Irrigation and PORALG.
>
> If the system is not delivered as agreed we will be forced to cancel the
> training and the government will have to incur the costs.
>
> Thank you.
>
> Chris
>
> *Chris Brown*
> WEL Group
> Tel: +255(0) 785 894711
> Tel: +44(0) 7772 421562
> Skype: chris.brown7
>
> On 08 June 2018 at 14:18 Ismail Yusuf <ismailkoleleni@xxxxxxxxx> wrote:
>
> Hello Chris,
>
> John will get back to you on this shortly.
>
> Thanks
>
> On Thu, Jun 7, 2018 at 3:24 PM Chris Brown < chris.brown@xxxxxxxxxxxxxx>
> wrote:
>
> Dear Ismail,
>
> I hope you are well.
>
> Can you please share the link to the development site so that the CDMT can
> review the data upload progress and prepare the necessary materials for the
> training which is due to start on the 18th June.
>
> Many thanks,
>
> Chris
>
> *Chris Brown*
> WEL Group
> Tel: +255(0) 785 894711
> Tel: +44(0) 7772 421562
> Skype: chris.brown7
>
> On 01 June 2018 at 15:20 Ismail Yusuf < ismailkoleleni@xxxxxxxxx> wrote:
>
> Dear Chris,
>
> We have been working thoroughly with the inventory and hierarchy file. New
> findings were observed, especially with Hierarchy data file. We have also
> noted, in the inventory data validation columns, someone was changing
> concatenated heirarchy column we were using for validation, replacing
> hierarchy names with existing formula instead of actually updating
> Region/LGA/Ward/Village columns.
>
> Till now, about 94,984 DPs have no trouble after flagging out the new
> issues. Attached file has more detailed description of new findings.
>
> Re your sub-villages question, it is very challenging to include
> sub-village both as org unit hierarchy or as a static indicator. Both
> options will require aligning the existing DP data to sub village (it is
> even more complex having as a static indicator because, list of options for
> this static indicator will have to depend which village a selected DP is
> in).
>
> A lot of thinking and Interface re-design may be required to achieve this.
>
> I suggest, we concentrate to getting the current task of data upload
> finished, then we can start thinking of this and other features that can
> come up.
>
> Regards,
>
> On Fri, Jun 1, 2018 at 1:59 PM Chris Brown < chris.brown@xxxxxxxxxxxxxx>
> wrote:
>
> Dear Ismail,
>
> I hope you are well.
>
> I just wanted to check in and see how things are progressing at your end,
> and to ask a quick question.
>
> The ministry has raised the question about putting the subvillage data
> into the WPDM. I explained the challenges with the hierarchy to them, but
> was wondering if it might be possible to put the subvillage in as a static
> indicator. Do you think that might be possible going forward?
>
> Best wishes,
>
> Chris
>
> *Chris Brown*
> WEL Group
> Tel: +255(0) 785 894711
> Tel: +44(0) 7772 421562
> Skype: chris.brown7
>
> On 30 May 2018 at 18:37 Ismail Yusuf < ismailkoleleni@xxxxxxxxx> wrote:
>
> Dear Chris,
>
> Thank you for following this up.
>
> We now have access to the server.
>
> Regards,
>
> On Wed, 30 May 2018, 09:17 Chris Brown, < chris.brown@xxxxxxxxxxxxxx>
> wrote:
>
> Dear Ismail,
>
> Apologies for the access issues. I notified the head of the ICT department
> here in the Ministry yesterday and have been assured it will be acted on
> today.
>
> Could you try accessing the development server again this morning and
> periodically throughout the day and notify me if you still do not have
> access this afternoon.
>
> Thanks & sorry again for the inconvenience.
>
> Chris
>
> *Chris Brown*
> WEL Group
> Tel: +255(0) 785 894711
> Tel: +44(0) 7772 421562
> Skype: chris.brown7
>
> On 29 May 2018 at 13:38 Ismail Yusuf < ismailkoleleni@xxxxxxxxx> wrote:
>
> Dear Chris,
>
> Thank you, we will proceed as advised by Dr. Kissaka.
>
> On a separate note, we have just realised our access to the WPDM server we
> configured at eGA has been removed. We can not do anything in the back end
> of the system. Data upload requires accessing the system back end. Can you
> please follow up with the ICT team to find out why?
>
> This may delay data upload process, well appreciate quick follow up.
>
> Thanks.
> Ismail
>
> On Mon, May 28, 2018 at 2:44 PM, Chris Brown < chris.brown@xxxxxxxxxxxxxx>
> wrote:
>
> Dear Dr. Kissaka & Ismail,
>
> If this is going to be the easiest way to resolve this situation, then let
> us proceed in the way you suggest, and we will then find a way to enter the
> remaining 2400 DP's manually via the CDMT and DWE network.
>
> Thank you for your continued understanding and co-operation.
>
> Kind regards,
>
> Chris
>
> *Chris Brown*
> WEL Group
> Tel: +255(0) 785 894711
> Tel: +44(0) 7772 421562
> Skype: chris.brown7
>
> On 28 May 2018 at 12:17 MMkissaka < mkissaka@xxxxxxxxx> wrote:
>
> Dear Chris and Ismail,
>
> We have worked hard to reach this point. I recommend that the Client and
> CDMT to accept to work on the remaining 2000+ water points instead of
> uploading as they are otherwise it will be difficulty to safely close this.
>
> Rgds
> Kissaka
>
>
>
> On Monday, May 28, 2018, 11:45:43 AM GMT+3, Ismail Yusuf <
> ismailkoleleni@xxxxxxxxx> wrote:
>
>
> Dear Chris,
>
> Your go ahead is well noted.
>
> Re: uploading DPs with blank, this is possible however I should be noted
> that data visualisation in the dashboard will show statistics that does not
> tally with total number of DPs in the system. This has always been pointed
> out as a system functionality problem, that is why we want to ensure we
> upload only DPs with complete data.
>
> Kindly let us know if you still want DPs with blank data uploaded they way
> they are. That way, we can not guarantee tallying of data displayed in
> Dashboard and reports with total number of DPs in the system.
>
> Regards,
>
> On Mon, May 28, 2018 at 10:32 AM, Chris Brown < chris.rm.brown@xxxxxxxxx>
> wrote:
>
> Dear Ismail,
> Thanks for your quick response this morning.
> I would like to request that you go ahead with the upload as outlined in
> your email.
> However regarding the 2000+ DPs that come up as a non match in your
> validation sheet, these are all due to one or more indicators being blank.
> As far as we can see there are no longer any data mismatches.
> Are these DPs not able to be uploaded in their existing form? This would
> allow the CDMT to then work with the DWEs to fill the remaining gaps.
> If the hierarchy and indicator lists match for all the present indicators
> can you out advise on whether this would be possible?
> Many thanks,
> Chris
>
> On Mon, May 28, 2018, 9:59 AM Ismail Yusuf < ismailkoleleni@xxxxxxxxx>
> wrote:
>
> Dear Chris,
>
> This is to confirm receipt of data file.
>
> From the quick analysis of the inventory file, below is the summary and
> recommendations.
>
> - The Inventory contains a total of 97,470 distribution points, to which
> 95,059 DPs have passed the validation and 2,441 DPs still have validation
> problem.
> - That makes 98% of the DP passing validation.
> - We can go ahead filter out the 2,441 DPs with trouble and only work on
> uploading 95,059. The rest can be entered manually by CDMT/DWE after
> training engagement. * We will need your go ahead (signing off) on this.*
> - We recommend the CDMT & MoWI not to make any updated in the inventory,
> until we upload data in the system, there after all data management will be
> done via system.
> - We will wipe out all hierarchy and indicator meta-data and options and
> update exactly as they appear in the validation sheet.
>
> We will need a week to upload everything. If you give a go ahead, you
> should expect to hear from us early next week for your side to confirm if
> everything is okay in the system.
>
> Thanks, and looking forward to hear from you.
>
> Regards,
>
> On Sun, May 27, 2018 at 5:14 PM, Chris Brown < chris.brown@xxxxxxxxxxxxxx>
> wrote:
>
> Dear Ismail,
>
> I hope you are well and have had a restful weekend.
>
> Apologies again for the CDMT missing the deadline for Wednesday, Thursday
> and Friday last week.
>
> We had the same recurrent basic cleaning issues as over the past few
> months and the team grappled with the file size which slowed things down a
> lot.
>
> We've been working on the resolving the issues over the weekend and I
> think they are now sorted out.  The only issues you should find with the
> validation now is from gaps in the data which we obviously can't do
> anything about until we engage DWE's during the training.
>
> I have sent you a separate weTransfer link message with the data file
> attached - please could you confirm receipt when you get it, just so that I
> know you have received it okay.
>
> Thanks & best wishes,
>
> Chris
>
>
> *Chris Brown*
> WEL Group
> Tel: +255(0) 785 894711
> Tel: +44(0) 7772 421562
> Skype: chris.brown7
>
> On 25 May 2018 at 05:28 Ismail Yusuf < ismailkoleleni@xxxxxxxxx> wrote:
>
> Dear Chris,
>
> Good morning!
>
> Any updates on the clean data?
>
> Looking forward to hear from you.
>
> On Tue, May 22, 2018 at 4:59 PM, Ismail Yusuf < ismailkoleleni@xxxxxxxxx>
> wrote:
>
> Dear Chris,
>
> Is this completed region Ruvuma? If you intended to send data only to
> Ruvuma regions, it is fine.
>
> I have noticed a lot of empty rows highlighted in Red. Was that on purpose?
>
> Its okay, you can send on Thursday when it is fully completed.
>
> Regards,
> Ismail
>
> On Tue, May 22, 2018 at 12:46 PM, Chris Brown < chris.brown@xxxxxxxxxxxxxx>
> wrote:
>
> Dear Ismail,
>
> Thank you again for sending the template through.
>
> Attached is a completed region done by the CDMT - would it be possible for
> your team to check this just to make sure that everything is in order?
>
> We expect the CDMT to complete 95% of the data clean by tomorrow as agreed
> - the remaining 5% will be completed by end of day Thursday. Apologies for
> the delay in this 5% but it is due to lack of electricity at the Ministry
> which has hindered the ability of the team to complete this work in a
> timely fashion.
>
> Kind regards,
>
> Chris
>
> *Chris Brown*
> WEL Group
> Tel: +255(0) 785 894711
> Tel: +44(0) 7772 421562
> Skype: chris.brown7
>
> On 18 May 2018 at 10:28 Ismail Yusuf < ismailkoleleni@xxxxxxxxx> wrote:
>
> Hi Chris,
>
> I have shared Tanzania Inventory.xlsx
> <https://drive.google.com/file/d/1p6VP6HvsZqQ8OKywFLBexmDFFOeYHOI6/view?usp=drive_web>
>  file as a google drive link, you need to download from that link since the
> file is big can not be attached via email.
>
> The file contains sheets with combine inventory, and validation sheets.
>
> Please confirm if you have seen the same.
>
> Regards,
>
> On Fri, May 18, 2018 at 12:14 PM, Chris Brown < chris.brown@xxxxxxxxxxxxxx>
> wrote:
>
> Dear Ismail,
>
> Thank you again for taking the time to come in yesterday and work with the
> team on the data issues.
>
> Apologies for not getting back to you sooner but we have had no
> electricity at the Ministry until now.
>
> I agree with all of the points in your email and the team will start
> working on it immediately.
>
> Looking at the file you have sent through, this seems to be just the
> hierarchy list and not the combine data file with the validation checks
> that you showed the team yesterday. Would it be possible for you to send
> that file through asap so the team can start using it as you instructed
> yesterday.
>
> Due to the loss of time from the lack of electricity, the team is asking
> if John could come in on Monday instead, when they have had some time to
> work on the data.
>
> Thanks & best wishes,
>
> Chris
>
>
> *Chris Brown*
> WEL Group
> Tel: +255(0) 785 894711
> Tel: +44(0) 7772 421562
> Skype: chris.brown7
>
> On 17 May 2018 at 22:46 Ismail Yusuf < ismailkoleleni@xxxxxxxxx> wrote:
>
> Dear Chris,
>
> As discussed during the meeting today. Please find attached combined
> country wide combine data file with Validation checks.
>
> - Basically we are validating each indicator to ensure data aligns with
> the indicator provided option. We have added validation column for each
> validated indicator for the team to easily track which indicator needs
> correction.
> - We have also validated against hierarchy to ensure the hierarchy
> indicated in the data is exactly the same and matches the hierarchy file
> provided.
> - About 50% of provided DP data has at least one indicator not aligned
> with defined option.
> - We recommend the team to make update on this inventory and share back to
> us the corrected file ready for upload.
> - For the hierarchy, the team should clean up all unwanted hierarchy. We
> will upload exactly what is uploaded, therefore there is no need to colour
> code anything. If there is duplicate, they can clean up all of them and
> remain with only correct data.
>
> The team requested us to pay a visit tomorrow, John Mukulu will come over
> in the afternoon and see if they are progressing well as well as provide
> technical support if needed.
>
> As agreed, we expect to receive clean data by Wednesday 23 May 2018, we
> will try and get data uploaded in the system by 31 May 2018.
>
> Feel free to let us know if you/the team have any questions.
>
> Kind Regards,
> Ismail
>
> ​
>  Tanzania Inventory.xlsx
> <https://drive.google.com/file/d/1p6VP6HvsZqQ8OKywFLBexmDFFOeYHOI6/view?usp=drive_web>
> ​
>  CST Indicator list April 2018_Revised
> <https://docs.google.com/spreadsheets/d/1wtZRyMIv1iOBSNyvZW0ck5ug5fw8C6Ug9MQIfPNblBU/edit?usp=drive_web>
> ​
>
>
>
>
> --
> Ismail Yusuf Koleleni,
> Computer Science and Engineering Department
> College of Information and Communication Technology (CoICT)
> University of Dar es Salaam
> Skype: e_smile1
>
>
>
>
> --
> Ismail Yusuf Koleleni,
> Computer Science and Engineering Department
> College of Information and Communication Technology (CoICT)
> University of Dar es Salaam
> Skype: e_smile1
>
>
>
>
> --
> Ismail Yusuf Koleleni,
> Computer Science and Engineering Department
> College of Information and Communication Technology (CoICT)
> University of Dar es Salaam
> Skype: e_smile1
>
>
>
>
> --
> Ismail Yusuf Koleleni,
> Computer Science and Engineering Department
> College of Information and Communication Technology (CoICT)
> University of Dar es Salaam
> Skype: e_smile1
>
>
>
>
> --
> Ismail Yusuf Koleleni,
> Computer Science and Engineering Department
> College of Information and Communication Technology (CoICT)
> University of Dar es Salaam
> Skype: e_smile1
>
>
>
>
>
>
>
> --
> Ismail Yusuf Koleleni,
> Computer Science and Engineering Department
> College of Information and Communication Technology (CoICT)
> University of Dar es Salaam
> Skype: e_smile1
>
>
>
> --
> Ismail Yusuf Koleleni,
> Computer Science and Engineering Department
> College of Information and Communication Technology (CoICT)
> University of Dar es Salaam
> Skype: e_smile1
>
>
>
> --
> Ismail Yusuf Koleleni,
> Computer Science and Engineering Department
> College of Information and Communication Technology (CoICT)
> University of Dar es Salaam
> Skype: e_smile1
>
>
>
> --
> Ismail Yusuf Koleleni,
> Computer Science and Engineering Department
> College of Information and Communication Technology (CoICT)
> University of Dar es Salaam
> Skype: e_smile1
>

References