widelands-dev team mailing list archive

[kaputtnik <somal@xxxxxxxx>]

It's hard to explain what i meant what is happend to trigger the error. Fell free to ask if something is unclear...

All following is related to widelands.org, not to the alpha-sites.

Fact is:
1. All images which was uploaded in year 2009 have the faulty path (".../django_projects/widelands/media//wlimages...")
2. All images which where uploaded in 2010 and afterwards have the right path (".../django_projects/wlwebsite/code/widelands//media//wlimages...")

Note the additonal "wlwebsite/code" part on images uploaded in 2010 and afterwards.

I assume that the website has changed the destination on the server somewhen in this period. On the admin site ( https://wl.widelands.org/admin/wlimages/image/?all= ) we see the last image upload in 2009 is on May 28, 2009. The next image upload was on Jan. 18, 2010. The one in 2009 contains the wrong path, the latter one contains the right path. So somewhen inbetween this period (May 28, 2009 and Jan. 18, 2010) the destination change of the website path was made.

How does the destination change affect the images? This is all made in function create_and_save_image() in our ImageManger http://bazaar.launchpad.net/~widelands-dev/widelands-website/trunk/view/head:/wlimages/models.py#L33 The "wrong" thing is happend here:

>        path = "%swlimages/%s" % (MEDIA_ROOT,image.name)
>        url = "%swlimages/%s" % (MEDIA_URL,image.name)
>
>        destination = open(path,"wb")
>        for chunk in image.chunks():
>            destination.write(chunk)
>
>        im.image = path
>        im.url = url

Especially the forlast line is faulty. The "path" contains the absolute path to the image including the image filename. So for images uploaded in 2009 it is something like this:

/var/www/django_projects/widelands/media//wlimages/Saledus.png

For files which are uploaded in 2010 and afterwards it looks like this:

/var/www/django_projects/wlwebsite/code/widelands//media//wlimages/SuperMan.png 

In the above line "im.image = path" the image is of type FileField which is restricted to 100 chars and has only one required argument "upload_to=". See https://docs.djangoproject.com/en/1.3/ref/models/fields/#django.db.models.ImageField Our model do only use the "upload_to" argument and do not provides fields for the height_field and width_field. To obtain these values (also the filesize) we have to use the fileobject instead. See https://docs.djangoproject.com/en/1.3/ref/files/file/

The File class has some usefull attributes like "name" and "size". The ImageFile class have the additional attributes "width" and "height".

So, what happend if we try to obtain the width, height or size of an image by editing an wikiarticle? Django trys to get the fileobject and checks if the path is valid (this is made in /lib/python2.7/site-packages/django/core/files/storage.py and /lib/python2.7/site-packages/django/utils/_os.py function safe_join() ) The related values for images uploaded in 2009 are:

In /lib/python2.7/site-packages/django/core/files/storage.py funtion path():

self.location = The current location ("/var/www/django_projects/wlwebsite/code/widelands//media/")
name = The name of the file relative(!) to MEDIA_ROOT (currently stored with "im.image = path" as absolute(!) path to the image. For above example it is "/var/www/django_projects/widelands/media//wlimages/Saledus.png")

In /lib/python2.7/site-packages/django/utils/_os.py function safe_join():

base_path = "/var/www/django_projects/wlwebsite/code/widelands/media/" ( self.location )
final_path = "/var/www/django_projects/widelands/media//wlimages/Saledus.png"

Because the final_path do not startswith base_path the value error is triggerd and we see the "SuspiciousOperation" error.

To get rid of the error, we have to do:

1. Change the code which stores the filename
from:
im.image = path

to:
im.image = "wlimages/%s" % (name)

This will correct future uploads to the website. To correct the past Image entries, we have to do step 2:

2. Change the entries for field "image" in the database table wlimages. On my local copy this could be simple done. I dont know of this usecase could be used on the server too. For my local installation i could do:

2.a. get a dumpdata for table wlimages:
> ./manage.py dumpdata --indent=2 > wlimages.json (The indent-option is only for convenience )

2.b. Change all(!) values for field "image" in the dumpdata:
from (f.e.):
"image": "/var/www/django_projects/widelands/media//wlimages/Saledus.png"

to:
"image": "wlimages/Saledus.png"

In words: Delete all chars before "wlimages/image.extension" including the leading slash(es) before "wlimages"

2.c. Reload the edited table "wlimages.json" into the database:
> ./manage.py loaddata wlimages.json
The output should be like "Installed X object(s) from X fixture(s)" where X is the number of datasets. 

Thats all. If you reactivate alpha.widelands.org (currently i get a 502 "Bad Gateway" error) we could maybe test this over there?

At last: Changing the value for field "image" as explained, will also correct an error on the admin page: if you open https://wl.widelands.org/admin/wlimages/image/166/ and click on the link in the right of "Image", a "Page not found" error occurs. With my modifications a click on that link will present the image.

Hopefully i didn't confuse you again... 

A different question: I couldn't update this branch because i get allways a "diverged" hint. What is the way to update this branch?
-- 
https://code.launchpad.net/~franku/widelands-website/handle_big_images/+merge/247235
Your team Widelands Developers is subscribed to branch lp:widelands-website.