libravatar-fans team mailing list archive

[Lars Kruse <devel@xxxxxxxxxxxxx>]

Hello,

we finished our weekly IRC meeting a few hours ago.

The next meeting will happen on the 16th of September at 19:00 UTC.
Please join it, if you are interested!

Attached you find the full log of today's meeting.
The wiki contains a summary:
  https://wiki.libravatar.org/shutdown-coordination/?updated#index2h2

We discussed about:
* the current state of the new implementation
* data migration and why we skip passwords
* pre- and post-migration announcements
* handling of the Mail-Domain @libravatar.org
* handling of backups

Enjoy the details!

Cheers,
Lars

21:18 <sumpfralle> Ok - welcome to the fourth IRC meeting!
21:18 <falko> first things first: sorry for not being around the last weeks (and meetings), but I really had to take the time off before starting the new job.
21:18 <sumpfralle> Welcome back!
21:18 <falko> merci
21:18 <sumpfralle> Which topic should we start with?
21:19 <falko> status of dev?
21:19 <sumpfralle> yeah!
21:21 <falko> so. I've seen the new design. it's not bad, but I'm not a huge fan of the startpage.
21:21 <falko> Anyway, I merged it into my tree and only fixed a few tests that failed.
21:21 <sumpfralle> Cool!
21:21 <sumpfralle> Let's progress incrementally ...
21:22 <falko> I'd love to show it, but OpenShift Online currently has some issues and therefore it doesn't deploy.
21:22 <nipos> The start page was what we discussed here as preview.Everyone likes it if I remember correctly
21:22 <nipos> *liked
21:22 <sumpfralle> There has never been a design, that _everybody_ liked :)
21:23 <sumpfralle> nowhere :)
21:23 <falko> I'll ping @clime tomorrow to see if he's around and can deploy it somewhere on OpenStack, as we discussed a few days ago.
21:23 <sumpfralle> falko: so you are planning to use the Fedora infrastructure for getting your code ready?
21:24 <falko> Nah. Design is fine more or less. If most ppl like it, it's ok.
21:24 <sumpfralle> i.e. we will keep the current instance on the host sponsored by ij and move to Fedora with the shift to your code?
21:25 <falko> @sumfralle: @clime tried it with Fedora OpenShift, but failed (please don't ask the exact reasons - I really don't remember). Now he thinks he can do it on Fedora OpenStack and I'm completely fine with that.
21:25 <sumpfralle> what is the difference?
21:25 <sumpfralle> (technically)
21:26 <falko> Yes, if it's fine with @ij_ to stay with his sponsored instance for the moment. If not, we can (more or less) move to AWS at any time (costs on me...).
21:27 <falko> @sumpfralle you're testing me. right? :-) OpenShift is the Kubernetes platform, while OpenStack is the IaaS platform (providing VMs so to say).
21:27 <falko> A total usual setup would be to run OpenShift _on_ OpenStack.
21:28 <sumpfralle> no, I am not involved with cloud infrastructure - I am used to administrating in the old-fashioned way
21:28 <falko> @sumpfralle did that for the past 20 years. :-)
21:29 <sumpfralle> How would you describe the current state of your development?
21:29 <sumpfralle> Where do you see missing pieces?
21:29 <falko> Anyway. This way (using OpenStack) it's easier for @clime to give (VM root) access to co-maintainers.
21:30 <nipos> Two links in the header nav link to nowhere (todo).That's all I remember from my local instance
21:30 <sumpfralle> root for co-maintainers: cool!
21:31 <falko> Current state of dev. Uh... I wanted to get it deployed in the last 2 days to see if it's running fine on mobile as well, but it looks like I have to test it with software (browser) instead of real (android and ios) hardware phones.
21:31 <falko> Yes, two links are still deadlinks, since I didn't write the pages yet - will do that with the next 1 - 2 weeks.
21:31 <falko> The size parameter should work now. Including s= and s=0 which caused errors.
21:32 <sumpfralle> size: for image delivery?
21:32 <falko> yes
21:32 <nipos> I tested many parts of the new design in a small browser window.It's optimized to work well there.
21:32 <sumpfralle> great
21:32 <sumpfralle> falko: could you give us a short introduction to the technical pieces, that your code does now?
21:33 <sumpfralle> (I do not know the internals, API and things ...)
21:33 <falko> @nipos: Great!!! As mentioned, I didn't have time check it out yet.
21:34 <sumpfralle> (I am taking notes on https://pad.riseup.net/p/libravatar-future-irc)
21:35 <falko> Ad technical. It's more or less the same basic idea as it used to be on libravatar. Same API. s= for size and d=<url> for the default image if the server doesn't find anything else
21:36 <falko> If no image is found it will return a redirect with the URL provided in d=
21:37 <falko> in regards to the internals. While libravatar saved all sizes of the avatar picture upfront in the file system, the new ivatar system saves the original picture BLOB in the database and returns the resized image on demand.
21:38 <falko> if this is too slow or causes too much CPU pressure, one can still implement caching.
21:38 <falko> (which would be a good idea anyway).
21:38 <sumpfralle> or do it in the webserver
21:38 <falko> yep. also an option.
21:38 <tleguern> varnish and nginx are here for that
21:39 <tleguern> I am doing the same as you, only keeping the original picture.
21:39 <falko> @tleguern you name it. cpus are much faster these days, compared when @fmarrier started his libravatar project.
21:39 <nipos> I would do it in the same way.That saves much storage
21:39 <falko> anyway.
21:41 <falko> this causes one major difference in regards to the old libravatar: Mirror servers are not easy to implement, as one would need to copy the database, instead of just rsync-ing the files.
21:42 <falko> But I didn't decide to go that route by accident. I was aware, that in the old days nobody wanted to start a mirror and I'm pretty sure this hasn't changed.
21:42 <nipos> Why not store the original image in file system.Then rsyncing is still possible and the mirror does only need an resize software.
21:42 <tleguern> Or they can rsync an export of the database, if someone wants to setup a mirror.
21:43 <nipos> Syncing all user data may be problematic because of security and GDPR
21:43 <sumpfralle> I recently stumbled upon the image resize module within nginx. This looked weird to me at first, but maybe it could be useful here. Anyway: the design is probably finished more or less, thus our discussion is futile? :)
21:44 <falko> Should we _really_ need this feature to open mirrors, we still can think about this. For the moment, I don't see a reason to implement it.
21:44 <sumpfralle> Fine for me.
21:45 <falko> @sumpfralle Does this answer your questions for the moment? Any further questions?
21:45 <sumpfralle> Almost :)
21:45 <nipos> I think we don't really need mirrors anymore.Fedoras servers are big enough
21:45 <falko> @nipos I think so too!
21:45 <sumpfralle> How do you envision the process of the data migration from the old instance?
21:46 <falko> @sumpfralle Excuse me, but can you tell me what was the outcome of the last GDPR/DSGVO discussion?
21:47 <nipos> We decided for ignore it until it becomes a problem
21:47 <sumpfralle> We felt, that the current situation is OK and we would not want to implement new processes or similar, until the need may arise.
21:47 <sumpfralle> yes :)
21:47 <falko> Ack. Sounds like my approach :-)
21:48 <sumpfralle> I am glad, that we came to this conclusion ...
21:49 <falko> So... In regards to data migration. I'd see it like that. We create export files for _all_ users - this includes their mail addresses and avatars. I'll write some import script, that will take the whole bunch of export files and import them. Only thing I don't want to import: The passwords. I'd like to see if the people create new passwords.
21:50 <falko> After a user has been imported, create a one-time-login link, send it to the user with the information, that he/she should login and change their password and check if the avatar picture is still fresh and in good shape :-)
21:50 <sumpfralle> Do you have a specific reason for this desire?
21:50 <nipos> We could generate random passwords and mail them to the users
21:50 <nipos> Or the links,that works,too :)
21:51 <falko> @nipos: One-Time-Login (like a password reset) or a random generated password - I don't care. We can vote on this.
21:51 <tleguern> It will remind people the project exists.
21:51 <tleguern> So it's a good point :)
21:51 <sumpfralle> :)
21:51 <falko> @tleguern Yep, upfront I'd like to send a mail to all users, that we'll do this, so they are aware and don't feel scammed.
21:53 <sumpfralle> For this mail, we will probably need to find a good approach, how to describe "us". Maybe next meeting ...
21:53 <falko> @sumpfralle Good point! Put it on the agenda please!
21:53 <sumpfralle> falko: Besides the publicity stunt of asking users for new passwords - do you have another reason?
21:55 <falko> a) We know which users are still active. If they change their password we know for sure these people are still alive. b) I want stronger password hashes.
21:55 <falko> c) if user isn't alive and we don't see hits on the avatar picture of that person, we can probably delete it.
21:56 <falko> mails that bounce => delete user.
21:56 <falko> that was d)
21:56 <sumpfralle> OK. Thank you.
21:56 <sumpfralle> I do not have an opinion about this. How about the others - what do you think?
21:56 <falko> you're welcome
21:56 <nipos> Stronger passwords are great
21:57 <tleguern> I agree too
21:57 <nipos> I don't know what was used in the old version but today we have very big and secure hashes like whirlpool and sha512 and we should use them
21:57 <sumpfralle> OK - so we do the import with all data except for the passwords. And we will do a "set your password mail to the users".
21:57 <sumpfralle> And one mail before the move.
21:58 <falko> that's my idea so far. i'm open for other ideas or enhancements to that approach.
21:58 --> fmarier (~francois@fsf/member/fmarier) hat den Channel #libravatar betreten
21:59 <sumpfralle> I am fine with it.
21:59 <falko> perfect.
22:00  * sumpfralle thinks, nipos and tleguern are preparing a long answer, each :)
22:01 <falko> lol
22:01 <nipos> No,I'm ok with that solution,nothin more to say on that topic
22:01 <tleguern> Neither, this solution is fine.
22:01 <sumpfralle> good!
22:01 <sumpfralle> Do you have other topics?
22:01  * fmarier apologizes for being late
22:01 <falko> I don't think so and we're already over the time.
22:02 <sumpfralle> fmarier: welcome!
22:02 <tleguern> Hello fmarier
22:02 <falko> @fmarier welcome.
22:02 <nipos> hello
22:02 <sumpfralle> ok - so let us meet again in two weeks - 2018/09/16?
22:02 <tleguern> I have one point but it can be discussed next time: Shouldn't we catch up with gravatar new options ?
22:02 <nipos> Other topics...
22:02 <sumpfralle> I put it to the list
22:03 <@fmarier> one quick topic from me
22:03 <nipos> There are two more on the list for infrastructure for today
22:03 <sumpfralle> go ahead!
22:03 <@fmarier> I have Google Apps setup for email on @libravatar.org
22:03 <tleguern> They added a f= knobs to force the default image to be served instead of the real one and I think it is great for people looking to implement libravatar on their website
22:03 <@fmarier> There are basically two questions associated with that:
22:04 <@fmarier> 1. should we continue to use Google Apps or does anybody want to run a mail server?
22:04 <falko> @tleguern : Can you open an issue / feature request at https://git.linux-kernel.at please?
22:04 <@fmarier> 2. who wants a @libravatar.org email account?
22:04 <tleguern> @falko of course
22:04 <nipos> I prefer an own server or any more trustworthy provider
22:04 <sumpfralle> same for me
22:04 <falko> @tleguern I can have a look at this then. But it sounds like a good idea.
22:05 <sumpfralle> The mail traffic is more or less about sending password mails - correct?
22:05 <tleguern> I agree with @nipos on this
22:05 <@fmarier> outgoing mail is already handled by the current server
22:05 <@fmarier> this is for incoming mail
22:06 <@fmarier> like dev@xxxxxxxxxxxxxx, tls@xxxxxxxxxxxxxx, support@xxxxxxxxxxxxxx, etc.
22:06 <@fmarier> if someone has a mail server that can forward to external addresses and create aliases, that's probably all we need
22:06 <falko> I already have some large enterprise customer with GApps, so although I'm not a huge GApps fan, I'd take it as it is. But on the other hand I run my mailserver anyway and we can move the libravatar mailstore there.
22:07 <sumpfralle> I could do the mail hosting (just forwarding).
22:07 <falko> @sumpfralle If you could do that I'd be happy to leave this to you :-)
22:07 <nipos> I do only have a raspberry with a webserver and a database so I'm not the right person for doing that
22:07 <sumpfralle> :)
22:08 <@fmarier> ok, sumpfralle and I can follow up via email
22:08 <sumpfralle> great
22:08 <falko> @nipos no, better not then :-)
22:08 <nipos> I vote for sumpfralles server
22:08 <falko> +1 for sumpfralles server
22:08 <sumpfralle> it is not mine - but my local tech collective
22:08 <falko> *libravatar.org to /me please :-)
22:09 <sumpfralle> hehe - we really need more spam traps :)
22:09 <falko> And please do some fancy anti-spam stuff!
22:09 <@fmarier> The other infrastructure topic I put down is backups.
22:09 <sumpfralle> not fancy, but good, I think
22:09 <falko> @sumpfralle Fine!
22:09 <falko> @fmarier You mean backups of the data?
22:10 <falko> the avatar/user data?
22:10 <@fmarier> server backup
22:10 <nipos> The most important question for backups: Does Fedora create backups automatically?If yes we wouldn't have to care about it
22:10 <@fmarier> that includes the database, but also the server config
22:10 <@fmarier> basically, anything that would be required to recreate the service from scratch without losing anything
22:11 <sumpfralle> Volume: 100G?
22:11 <@fmarier> No, it's way less than that. Probably less than 1GB.
22:12 <@fmarier> It's on my personal S3 account right now. Anybody knows how to check the size of an S3 bucket?
22:12 <falko> Config is not an issue, since this will be automatically created (OpenStack, OpenShift, whatever...).
22:12 <falko> Ad size of S3 bucket) No idea :-|
22:13 <falko> Dunno the current backup strategy of Fedora OpenStack, but probably it's up to the "user" of the system.
22:13 <@fmarier> at the very least, the DB will need to be backed up
22:13 <falko> But my plan is to run a job that dumps the data(base) on regular basis to some other location (S3 is a good idea).
22:14 <@fmarier> anyways, I'm happy to keep hosting the backups until we migrate to the new server
22:14 <sumpfralle> If it is just that small (<50g), I could provide the place. The implementation needs a bit of thought (only encrypted, I guess).
22:14 <@fmarier> the backups are currently encrypted
22:14 <sumpfralle> something like borgbackup of the filesystem and the latest data dump?
22:14 <sumpfralle> good
22:14 <@fmarier> and the db backups are also encrypted to my GPG key
22:14 <tleguern> (https://git.linux-kernel.at/oliver/ivatar/issues/6)
22:14 <@fmarier> it's using duplicity at the moment
22:14 <sumpfralle> Good.
22:15 <sumpfralle> So we can discuss that, too.
22:15 <sumpfralle> (via mail)
22:15 <@fmarier> it would be trivial for me to point the backups to an ssh account somewhere
22:16 <sumpfralle> yes, that would be my approach, too. We will manage.
22:16 <falko> @tleguern thx
22:16 <@fmarier> I was slightly wrong, the backups are currently 6.2 GB.
22:16 <falko> @fmarier Do you want to get rid of the S3 soon?
22:16 <@fmarier> Probably because I keep a few days worth or something.
22:16 <falko> @fmarier Still not much.
22:17 <sumpfralle> yes, still fine
22:17 <@fmarier> If we have a better place to put them, moving away from S3 would be ideal.
22:17 <falko> @fmarier i see.
22:17 <sumpfralle> you send me your/the key - I will send the details.
22:17 <falko> @sumpfralle So you take the backups for the moment?
22:17 <@fmarier> ok, thanks sumpfralle, let's do this via email too.
22:17 <sumpfralle> yes, that was my offer
22:18 <sumpfralle> good
22:18 <sumpfralle> so we finished all interesting topics?
22:18 <falko> great.
22:18 <@fmarier> what about the GPG key to encrypt the DB backups to?
22:18 <sumpfralle> we will discuss that next time :)
22:18 <@fmarier> it might be good to have another person or two who could decrypt those backups
22:18 <@fmarier> right now, it's just me.
22:18 <sumpfralle> yes, of course
22:18 <sumpfralle> 2+
22:18 <@fmarier> That's also easy to add to my backup script.
22:18 <sumpfralle> well prepared for companions! :)
22:19 <falko> Let's discuss the GPG topic via mail, ok?
22:19 <@fmarier> sounds good