libravatar-fans team mailing list archive

[clime <clime7@xxxxxxxxx>]

Hey, I have finished the port. It looks stable now. No errors so far.
I think the important point was specifying multiple processes (4)
which toggled "multiprocess" switch on but just guessing here. Right
now, the configuration is 4 processes (corresponding to number of
cores) with 3 threads each (cause some threads might be waiting on
IO).

Should we enable http2 for ssl? I think we are not currently enabling
it. Do we have some benchmark for avatar loading? @Oliver Falk i think
you had something a long time ago - it would be great to try on
http1.1 -> http2 change.

Also I suggest we add Gandi (our domain provider) as our supporter as
they have been great with providing us the libravatar domains
(libravatar.org and libravatar.com) for free.

Best Regards
clime

On Sun, 14 Mar 2021 at 09:45, Oliver Falk <oliver@xxxxxxxxxxxxxxx> wrote:
>
> Hey!
>
> Thanks for your additional investigations and finding a possible solution!
> Looking forward to see if it‘s going well upon the next try! ;-)
>
> clime++ !!
>
> Oliver
>
> clime <clime7@xxxxxxxxx> schrieb am So. 14. März 2021 um 10:43:
>>
>> On Sat, 13 Mar 2021 at 23:35, Oliver Falk <oliver@xxxxxxxxxxxxxxx> wrote:
>> >
>> > Hey!
>> >
>> > Sorry, I’ve been on the road all day.
>> > I’ll also try to find time to investigate in the next days!
>>
>> Ok, it seems that only that knows how to trigger the 500 error (the
>> one with pscypg2)
>> is YisouSpider but his last try worked correctly (200) after I
>> somewhat tweaked mod_wsgi settings.
>>
>> I will probably try to do the port again this evening. It seems to be
>> something related to
>> how mod_wsgi handles threads and python interpreter context but it
>> didn't happen with older python
>> version (now python3.9, before python3.7).
>>
>> I will try again today around 9pm UTC if there are no errors in logs
>> from bots on the new server.
>> I hope either the current set or some other set of mod_uwsgi settings
>> should produce a stable
>> behavior.
>>
>> Best regards
>> clime
>>
>>
>> >
>> > Oliver
>> >
>> > clime <clime7@xxxxxxxxx> schrieb am Sa. 13. März 2021 um 23:21:
>> >>
>> >> On Sat, 13 Mar 2021 at 21:02, clime <clime7@xxxxxxxxx> wrote:
>> >> >
>> >> > Hey,
>> >> >
>> >> > during my testing, everything looked fine but as soon as I made a
>> >> > switch, I started to get 500 errors. Probably some problems with wsgi
>> >> > configuration. Might be related to psycopg2 module.
>> >> >
>> >> > I switched back to the old IP and will try to fix the problem. I will
>> >> > keep libravatar.org in maintenance mode for now and will try to find
>> >> > the problem.
>> >>
>> >> I have resumed normal operation on the old server. Sadly, I haven't
>> >> figured it out so far. The error seems to occur only non-deterministically
>> >> and probably is load-dependant (given that I started to get it heavily
>> >> after switch).
>> >>
>> >> There are two relevant error messages:
>> >>
>> >> [Sat Mar 13 22:01:57.242462 2021] [wsgi:error] [pid 8270:tid 8418]
>> >> [remote 106.11.156.69:18651]   File
>> >> "/usr/lib64/python3.9/importlib/__init__.py", line 127, in
>> >> import_module
>> >> [Sat Mar 13 22:01:57.242467 2021] [wsgi:error] [pid 8270:tid 8418]
>> >> [remote 106.11.156.69:18651]     return
>> >> _bootstrap._gcd_import(name[level:], package, level)
>> >> [Sat Mar 13 22:01:57.242475 2021] [wsgi:error] [pid 8270:tid 8418]
>> >> [remote 106.11.156.69:18651]   File
>> >> "/data/venvs/libravatar/lib/python3.9/site-packages/django/db/backends/postgresql/base.py",
>> >> line 25, in <mo
>> >> dule>
>> >> [Sat Mar 13 22:01:57.242481 2021] [wsgi:error] [pid 8270:tid 8418]
>> >> [remote 106.11.156.69:18651]     import psycopg2 as Database
>> >> [Sat Mar 13 22:01:57.242488 2021] [wsgi:error] [pid 8270:tid 8418]
>> >> [remote 106.11.156.69:18651]   File
>> >> "/data/venvs/libravatar/lib64/python3.9/site-packages/psycopg2/__init__.py",
>> >> line 51, in <module>
>> >> [Sat Mar 13 22:01:57.242494 2021] [wsgi:error] [pid 8270:tid 8418]
>> >> [remote 106.11.156.69:18651]     from psycopg2._psycopg import (
>> >>               # noqa
>> >> [Sat Mar 13 22:01:57.242509 2021] [wsgi:error] [pid 8270:tid 8418]
>> >> [remote 106.11.156.69:18651] SystemError: initialization of _psycopg
>> >> raised unreported exception
>> >>
>> >> And then repeating:
>> >> [Sat Mar 13 21:41:33.310712 2021] [wsgi:error] [pid 7411:tid 7563]
>> >> [remote 45.91.21.5:18613] Traceback (most recent call last):
>> >> [Sat Mar 13 21:41:33.310739 2021] [wsgi:error] [pid 7411:tid 7563]
>> >> [remote 45.91.21.5:18613]   File "/data/wsgi.py", line 26, in <module>
>> >> [Sat Mar 13 21:41:33.310748 2021] [wsgi:error] [pid 7411:tid 7563]
>> >> [remote 45.91.21.5:18613]     application = get_wsgi_application()
>> >> [Sat Mar 13 21:41:33.310755 2021] [wsgi:error] [pid 7411:tid 7563]
>> >> [remote 45.91.21.5:18613]   File
>> >> "/data/venvs/libravatar/lib/python3.9/site-packages/django/core/wsgi.py",
>> >> line 12, in get_wsgi_application
>> >> [Sat Mar 13 21:41:33.310759 2021] [wsgi:error] [pid 7411:tid 7563]
>> >> [remote 45.91.21.5:18613]     django.setup(set_prefix=False)
>> >> [Sat Mar 13 21:41:33.310767 2021] [wsgi:error] [pid 7411:tid 7563]
>> >> [remote 45.91.21.5:18613]   File
>> >> "/data/venvs/libravatar/lib/python3.9/site-packages/django/__init__.py",
>> >> line 24, in setup
>> >> [Sat Mar 13 21:41:33.310772 2021] [wsgi:error] [pid 7411:tid 7563]
>> >> [remote 45.91.21.5:18613]     apps.populate(settings.INSTALLED_APPS)
>> >> [Sat Mar 13 21:41:33.310779 2021] [wsgi:error] [pid 7411:tid 7563]
>> >> [remote 45.91.21.5:18613]   File
>> >> "/data/venvs/libravatar/lib/python3.9/site-packages/django/apps/registry.py",
>> >> line 83, in populate
>> >> [Sat Mar 13 21:41:33.310784 2021] [wsgi:error] [pid 7411:tid 7563]
>> >> [remote 45.91.21.5:18613]     raise RuntimeError("populate() isn't
>> >> reentrant")
>> >>
>> >> Right now, I cannot reproduce the errors at all. I will continue the
>> >> investigation in the following days.
>> >> clime
>> >>
>> >> > clime
>> >>
>> >> _______________________________________________
>> >> Mailing list: https://launchpad.net/~libravatar-fans
>> >> Post to     : libravatar-fans@xxxxxxxxxxxxxxxxxxx
>> >> Unsubscribe : https://launchpad.net/~libravatar-fans
>> >> More help   : https://help.launchpad.net/ListHelp