← Back to team overview

brewtarget-devs team mailing list archive

Re: Database and XML

 

I knew that was coming, but the idea isn't so trusting of third party
sources. That is, we don't use data that some provider could use, and we
don't rely solely on the cloud in a way that could cause us to lose our
data. The database is always local, the source is open, and no one user has
recipes/ingredients that a company like GN could use in some nefarious way.
Basically, let 'em steal it. Worst case is, you just make more.

BTW, I work with GN directly every day. Nice guys, but the cloud is going
to kill them. They are no longer the only trusted source of metadata, and
they are trying to design a bunch of services to sell under the heading of
"this would be nice, you should pay for it". I think it's kinda funny.

For the near term, I like the idea of a version based on timestamp or some
such. Almost all people's computers now sync time to a trusted source, so
generically, you can trust it to be accurate close enough to make a version
string/hash work. Let me know what you wanted to do, but then you could
make it a binary choice to the user. If my local DB is newer than the
cloud, update the cloud. If the cloud is newer than my local, update my
local. You make a note upfront that the cloud back option isn't to let two
people work on the same content at the same time. In the short term, this
avoids the version issues you point out above while someone figures out a
good way of doing differential merges to SQLite databases (where I think
standard XML might make this easier). I don't want to use BeerXML at all.
But if we can store individual recipes remotely, and timestamp them somehow
(or version them somehow), then it becomes a bit easier to handle your 1,
1.1, and 1.2 scenario above.

I likely won't tackle undo with this. First version will just overwrite the
cloud file, and we'll see how good I am at getting it to use the logic in
load() to merge it back. Then I'll see what we can do about differential
updates.

-Pete


On Sun, Apr 13, 2014 at 12:13 AM, mik firestone <mikfire@xxxxxxxxx> wrote:

> Crap. I missed the "reply all". Adding the list back in.
>
> Yes. I do hate the spec. It is broken and reflects homebrewing and XML as
> we understood it 10 years ago. beersmith and almost every other homebrew
> software or site has pretty much abandoned the spec. It is time we did too.
>
> I don't know how beersmith v2 works. I use brewtarget, with some
> brewersfriend thrown in for water chemistry.
>
> Consider how you will determine two recipes (or hops, or fermentables or
> ...) are the same. The name is not a good solution, since names can be
> changed and I mistrust string comparisons. Maybe like a UUID (
> http://en.wikipedia.org/wiki/Universally_unique_identifier) or an OID (
> http://en.wikipedia.org/wiki/Object_identifier)? I kind of like the idea
> of an OID over a UUID, but OIDs are a pain to generate/maintain.
>
> Consider SQL as your interchange format. Watch out for this idea. There
> are many devils hiding in the details.
>
> Consider using builtin SQLite functionality (like an audit log or a replay
> log or a something).
>
> Consider leveraging a simple versioning system I intend on implementing in
> v2.2. I can explain my ideas on that so far if you want.
>
> Consider implementing the undo stack, and then figuring out how to
> "reverse" it to get a rebuild stack.
>
> Consider how you will do a three way merge -- that is, I change the recipe
> v1.0 on system A resulting in v1.1, I change recipe v1.0 on system B
> resulting in recipe v1.2 and now I want to merge all changes on system C.
> Maybe we find some way of leveraging git, or some other SCCS?
>
> And having said that, I'm going to shutup. I'm not a fan of clouds. They
> either evaporate (think dotcom) or suddenly you find all your data belongs
> to them (think gracenote). I don't want my general pessimism to either
> discourage you or prevent what could be a great feature and I'm just too
> old and cantankerous to see it.
>
> Mik
>
>
>
> On Sat, Apr 12, 2014 at 11:24 PM, Peter Buelow <goballstate@xxxxxxxxx>wrote:
>
>> OK then, I knew about the XML, but you seemed to hate it. I didn't know
>> about the HTML, but doesn't help much. The question isn't the specifics,
>> but what makes more sense on a cloud service? My thought was a differential
>> db, so we can share pieces easily (how is tbd), but also make the cloud
>> smart to always stay up to date with the (a) local copy. And it should be
>> able to sync such that my desktop copy is the same as my laptop, and
>> eventually, somehow, a mobile copy. If not XML, then what form? Its just a
>> file to Dropbox, so how to make a cloud service useful? Xml makes the most
>> sense, but no one likes it. SQLite is easiest, but not easily extendable.
>> This is where beersmith still isn't quite right in my mind. Close, but not
>> exactly.
>>
>> Does that make more sense? I don't want this to mimic Beersmith honestly.
>> I'd rather try to make it smart and flexible. If I choose to sync to the
>> cloud, then no matter what machine I use, I always see the same database.
>> But only if I ask to do that. I should also be able to upload individual
>> recipes.
>>
>> As I write this, I think its a two step approach. Sync for the entire db.
>> Using sqlite where Database() can reconcile to the cloud, plus the ability
>> to upload individual recipes as HTML or XML. Have some sort of right click
>> upload option.
>>
>> Make more sense now? I may have a better path now that I wrote this out.
>> But I think there are still issues to consider.
>>
>> -Pete
>> On Apr 12, 2014 9:07 PM, "mik firestone" <mikfire@xxxxxxxxx> wrote:
>>
>>> I'm confused.
>>>
>>> We already can export a recipe to XML. We can import them to. I have a
>>> memory that we aren't exporting brewnotes, which I would like to fix. We
>>> can export to HTML too.
>>>
>>> I use both methods all the time. Export to html so I always have access
>>> to the recipe via my phone at the lhbs, and xml exports for back up and
>>> sharing with friends.
>>>
>>> Mik
>>> On Apr 12, 2014 8:59 PM, "Peter Buelow" <goballstate@xxxxxxxxx> wrote:
>>>
>>>> Nah, I'm thinking something along the lines of what Beersmith does. You
>>>> can't really share your Brewtarget recipes in a similar way. Using Dropbox
>>>> or Google, it should be possible to upload individual recipes somehow (my
>>>> thinking with XML) such that you can share with the public, or with a set
>>>> of named users. I don't think Brewtarget can do the recipe website
>>>> Beersmith does, but that doesn't mean we can't provide backup and sharing.
>>>>
>>>> So I guess my question is, is there a simple way to upload just a
>>>> recipe in some form to be shared? I think the other side of this is to
>>>> simply backup the database, and that should work fine with the current
>>>> database implementation, just getting it remotely to import. As an example,
>>>> when we call load(), double check to see if there is an online storage
>>>> connection, and if so and enabled, we'll check it and see if we should
>>>> update. This presents problems too, so maybe it's a choice to import the
>>>> online version, or just ignore it and backup only? I want to make this work
>>>> relatively seamlessly, such that the user isn't presented with 1000
>>>> options, but I also want it to be flexible such that it's possible to do
>>>> all of the cloud storage fun that might be possible.
>>>>
>>>> Looking for thoughts. What might be the best option(s) here?
>>>>
>>>>
>>>> On Sat, Apr 12, 2014 at 4:35 PM, Philip Lee <rocketman768@xxxxxxxxx>wrote:
>>>>
>>>>> I'm not quite sure what you mean by individual recipes. You can
>>>>> definitely have a database where there is only one recipe in the recipe
>>>>> table. They aren't dependent on each other.
>>>>>
>>>>>
>>>>> On Sat, Apr 12, 2014 at 4:04 PM, Peter Buelow <goballstate@xxxxxxxxx>wrote:
>>>>>
>>>>>> So the statement is to store the entire SQLite database remotely, and
>>>>>> then open it and try to merge it once? I ask, because I like the way
>>>>>> Beersmith does it. They can store individual recipes, not the entire thing.
>>>>>> However, I think maybe small steps first, and then work out how to do it
>>>>>> better.
>>>>>>
>>>>>> Is it possible to store individual recipes in SQLite then?
>>>>>>
>>>>>>
>>>>>> On Fri, Apr 11, 2014 at 9:07 AM, Philip Lee <rocketman768@xxxxxxxxx>wrote:
>>>>>>
>>>>>>> We are doing database merges for updates. This allows us absolutely
>>>>>>> certainty when updating the ingredients and recipes that ship with
>>>>>>> brewtarget. The XML is absolutely painful and messy as a backend. Mik found
>>>>>>> out the BeerXML spec does not even conform to XML!
>>>>>>>
>>>>>>> As far as I'm concerned, the XML should only be used for
>>>>>>> interoperability with other software.
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Apr 10, 2014 at 10:13 AM, Peter Buelow <
>>>>>>> goballstate@xxxxxxxxx> wrote:
>>>>>>>
>>>>>>>> Sorry, hashing out old questions. How good is the import and export
>>>>>>>> of the DB content to XML right now? I'm implementing the Dropbox (and soon
>>>>>>>> Google) storage, but I don't want to just blindly copy the sqlite database.
>>>>>>>> What I'd like to do is improve the Database class to handle incoming XML
>>>>>>>> content such that it can compare and update the local database where
>>>>>>>> necessary. It would also then be able to export content out to XML on the
>>>>>>>> remote storage.
>>>>>>>>
>>>>>>>> Thoughts? I know BeerXML isn't great, but is this the best plan? Is
>>>>>>>> there a smarter way to handle this type of update?
>>>>>>>>
>>>>>>>> --
>>>>>>>> Pete
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mailing list: https://launchpad.net/~brewtarget-devs
>>>>>>>> Post to     : brewtarget-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>>> Unsubscribe : https://launchpad.net/~brewtarget-devs
>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Philip G. Lee
>>>>>>> rocketman768@xxxxxxxxx
>>>>>>>
>>>>>>> --
>>>>>>> Mailing list: https://launchpad.net/~brewtarget-devs
>>>>>>> Post to     : brewtarget-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~brewtarget-devs
>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Pete
>>>>>>
>>>>>> --
>>>>>> Mailing list: https://launchpad.net/~brewtarget-devs
>>>>>> Post to     : brewtarget-devs@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~brewtarget-devs
>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Philip G. Lee
>>>>> rocketman768@xxxxxxxxx
>>>>>
>>>>> --
>>>>> Mailing list: https://launchpad.net/~brewtarget-devs
>>>>> Post to     : brewtarget-devs@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~brewtarget-devs
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Pete
>>>>
>>>> --
>>>> Mailing list: https://launchpad.net/~brewtarget-devs
>>>> Post to     : brewtarget-devs@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~brewtarget-devs
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>
>
> --
> In a world of ninja v. pirate, I pilot a Gundam
>



-- 
Pete

References