yade-dev team mailing list archive

[Bruno Chareyre <bruno.chareyre@xxxxxxxxxxxxxxx>]

On Wed, 13 Nov 2019 at 20:33, Anton Gladky <gladky.anton@xxxxxxxxx> wrote:

>
> could you please send me a script, which is doing the download/upload?
>
>
Please forgive in advance my naive bash crap and see below.

*cd /tmp*
*touch etag_last*






*wget --server-response --spider
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
<https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages>
2>&1 | grep ETag > etag_remoteif ! cmp -s etag_remote etag_last; then
wget
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
<https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages>
  unzip download?job=pages    rm -rf /var/www/doc/*    mv /tmp/public
/var/www/doc    rm download?job=pages*
*    mv etag_remote etag_last*
*fi*



> Actually, when I do
>
> "wget
> https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
> "
>
> I get right now the error code 8.
>

As I type this email that target works again (log at the end of this
email), yet for some reason (it occured at least twice in the last months,
probably only for a couple hours the first time).



> Actually, if you use the symbol &&, it will not execute the next step,
> if the previous one failed.
>
> Something like:
>
> wget
> https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
> &&
> <https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages&&;>
> rm -rf /var/www/yade-dem.org && cp * /var/www/yade-dem.org



Sounds like the good fix, thanks for mentioning the syntax.
I think we will try that.
The question of why this happens remains but that's perfectly fine to
workaround it.
Cheers
Bruno




$ wget
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
--2019-11-13 17:55:13--
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
Resolving gitlab.com (gitlab.com)... 35.231.145.151
Connecting to gitlab.com (gitlab.com)|35.231.145.151|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2019-11-13 17:55:14 ERROR 404: Not Found.

$ wget
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
--2019-11-14 17:09:32--
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
Resolving gitlab.com (gitlab.com)... 35.231.145.151
Connecting to gitlab.com (gitlab.com)|35.231.145.151|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location:
https://gitlab.com/yade-dev/trunk/-/jobs/349059349/artifacts/download
[following]
--2019-11-14 17:09:34--
https://gitlab.com/yade-dev/trunk/-/jobs/349059349/artifacts/download
Reusing existing connection to gitlab.com:443.
HTTP request sent, awaiting response... 302 Found
Location:
https://storage.googleapis.com/gitlab-gprd-artifacts/5c/74/5c748985bd13444e59322d1dd55fa2a5091e45a49c1db75d94366184ad2a7fe9/2019_11_13/349059349/361212541/artifacts.zip?response-content-disposition=attachment%3B%20filename%3D%22artifacts.zip%22%3B%20filename%2A%3DUTF-8%27%27artifacts.zip&response-content-type=application%2Fzip&GoogleAccessId=gitlab-object-storage-prd@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx&Signature=JpHFPisbZHmwBGSHy%2Bu%2BDN0Qyi8TQN%2F2J5DbGjSJiuKKO8T8XzyrxjfbEncC%0AgNSHmubYDcLB%2FScSvvS8m%2BJt%2F%2BHkerujryO%2B4FQMAzcnxg2ptUTE2hqR3F43%0AuJk6zo5OsJ8IFvzzfH43VRvhPXEiC2NwbC0FB7jIqvw%2FNevlghADoRxvG7BK%0A0m9yaeoywDilR%2FDu%2BB6qUegllIyp8gvc5XrjyDortOuSUM7wegpRGQ5vMBS1%0Arxn%2F1AvTyeUDaSkbyMeLmx9XxHBJsFdlBAPHdQ1nVgh1GmGU7uoIku7895nd%0AhxoDIfe4ANq5Qdj6O7LgKhr%2FHugAgkyPJ2B8VDf5ZQ%3D%3D&Expires=1573748375
[following]
--2019-11-14 17:09:35--
https://storage.googleapis.com/gitlab-gprd-artifacts/5c/74/5c748985bd13444e59322d1dd55fa2a5091e45a49c1db75d94366184ad2a7fe9/2019_11_13/349059349/361212541/artifacts.zip?response-content-disposition=attachment%3B%20filename%3D%22artifacts.zip%22%3B%20filename%2A%3DUTF-8%27%27artifacts.zip&response-content-type=application%2Fzip&GoogleAccessId=gitlab-object-storage-prd@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx&Signature=JpHFPisbZHmwBGSHy%2Bu%2BDN0Qyi8TQN%2F2J5DbGjSJiuKKO8T8XzyrxjfbEncC%0AgNSHmubYDcLB%2FScSvvS8m%2BJt%2F%2BHkerujryO%2B4FQMAzcnxg2ptUTE2hqR3F43%0AuJk6zo5OsJ8IFvzzfH43VRvhPXEiC2NwbC0FB7jIqvw%2FNevlghADoRxvG7BK%0A0m9yaeoywDilR%2FDu%2BB6qUegllIyp8gvc5XrjyDortOuSUM7wegpRGQ5vMBS1%0Arxn%2F1AvTyeUDaSkbyMeLmx9XxHBJsFdlBAPHdQ1nVgh1GmGU7uoIku7895nd%0AhxoDIfe4ANq5Qdj6O7LgKhr%2FHugAgkyPJ2B8VDf5ZQ%3D%3D&Expires=1573748375
Resolving storage.googleapis.com (storage.googleapis.com)...
216.58.206.240, 2a00:1450:4007:817::2010
Connecting to storage.googleapis.com
(storage.googleapis.com)|216.58.206.240|:443...
connected.
HTTP request sent, awaiting response... 200 OK
Length: 18604798 (18M) [application/zip]
Saving to: ‘download?job=pages’

download?job=pages
100%[=============================================>]  17.74M  23.0MB/s
 in 0.8s

>
> Am Mi., 13. Nov. 2019 um 18:05 Uhr schrieb Bruno Chareyre
> <bruno.chareyre@xxxxxxxxxxxxxxx>:
>
> >
> > Hi there,
> > yade-dem.org is currently down while yade-dev.gitlab.io/trunk/ is not.
> >
> > The reason is that yade server is downloading the gitlab artifacts like
> this:
> > wget
> https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
> ,
> > but at the moment that url returns nothing. Hence empty website.
> >
> > Obviously our script should test the output of wget, to not replace the
> content by nothing. We can fix this. Even so, I would like to understand
> why the artifacts are not there. They usually are. Is it because they
> expired on gitlab? Is it because of a failed pipeline?
> > Maybe we should form another url to try and get the content of
> yade-dev.gitlab.io/trunk/ (if possibe) instead of checking out an
> artifact.
> > I don't have time to check more right now so in case someone has
> inspiration, let me know:
> > - if you know how to make wget conditional
> > - if you know a better target url
> >
> > Cheers
> >
> > Bruno
> >
> > --
>
>