launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #15213
[Merge] lp:~rvb/maas/generic-field-1.2 into lp:maas/1.2
Raphaël Badin has proposed merging lp:~rvb/maas/generic-field-1.2 into lp:maas/1.2.
Commit message:
Add a copy of Django 1.4 GenericIPAddressField class (and the required utilities); monkey patch Django to add the new field if the version of Django is 1.3.
Requested reviews:
MAAS Maintainers (maas-maintainers)
Related bugs:
Bug #1130232 in maas (Ubuntu): "Implement GenericIpAddressField in MAAS rather than django."
https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1130232
For more details, see:
https://code.launchpad.net/~rvb/maas/generic-field-1.2/+merge/149786
After thinking about a few ways to do this we came to the conclusion that using a monkey patch was the only reasonable solution to avoid trouble when upgrading and being able to cope with already installed packages from the stable ppa (see the discussion on the bug).
We still have the choice to fix this in the packaging or in the upstream code but I think it's best to fix this in the code:
- this way, this way the new code gets exercised when we run the test suite on a platform with dj 1.3 (which is what the 1.2 lander does).
- this introduces only very minor divergence with trunk: a new module in src/maasserver/dj14/ which is not likely to introduce conflicts when backporting stuff from trunk to 1.2 and a tiny and well focused change to src/maasserver/models/__init__.py.
= Notes =
This branch adds the GenericIPAddressField model field (and the required utilities) in src/maasserver/dj14/. I've split up the code in there to reflect how the code was structured in Django itself.
The monkey patch (in src/maasserver/models/__init__.py) needs to happen at the very beginning of this file so that the model modules that use the GenericIPAddressField class can find it in the right place.
= Testing =
I've tested that the test suite passes on a precise machine with a version of Django without the genericipadressfield.diff patch.
I've also tested a package created from this branch in the lab with success.
--
https://code.launchpad.net/~rvb/maas/generic-field-1.2/+merge/149786
Your team MAAS Maintainers is requested to review the proposed merge of lp:~rvb/maas/generic-field-1.2 into lp:maas/1.2.
=== added directory 'src/maasserver/dj14'
=== added file 'src/maasserver/dj14/__init__.py'
--- src/maasserver/dj14/__init__.py 1970-01-01 00:00:00 +0000
+++ src/maasserver/dj14/__init__.py 2013-02-21 10:28:40 +0000
@@ -0,0 +1,4 @@
+# This module is a backport of Django's GenericIPAddressField model
+# field [0] and all the required utilities. That field was introduced in
+# Django 1.4 and this allows to use it with Django 1.3.
+# [0]: http://goo.gl/4yxsv
=== added file 'src/maasserver/dj14/forms.py'
--- src/maasserver/dj14/forms.py 1970-01-01 00:00:00 +0000
+++ src/maasserver/dj14/forms.py 2013-02-21 10:28:40 +0000
@@ -0,0 +1,25 @@
+# flake8: noqa
+# Extract of Django 1.4's forms/fields.py file with modified imports.
+from django.core import validators
+from django.forms.fields import CharField
+from maasserver.dj14.ipv6 import clean_ipv6_address
+from maasserver.dj14.validators import ip_address_validators
+
+
+class GenericIPAddressFormField(CharField):
+ default_error_messages = {}
+
+ def __init__(self, protocol='both', unpack_ipv4=False, *args, **kwargs):
+ self.unpack_ipv4 = unpack_ipv4
+ self.default_validators, invalid_error_message = \
+ ip_address_validators(protocol, unpack_ipv4)
+ self.default_error_messages['invalid'] = invalid_error_message
+ super(GenericIPAddressFormField, self).__init__(*args, **kwargs)
+
+ def to_python(self, value):
+ if value in validators.EMPTY_VALUES:
+ return u''
+ if value and ':' in value:
+ return clean_ipv6_address(value,
+ self.unpack_ipv4, self.error_messages['invalid'])
+ return value
=== added file 'src/maasserver/dj14/genericipaddressfield.py'
--- src/maasserver/dj14/genericipaddressfield.py 1970-01-01 00:00:00 +0000
+++ src/maasserver/dj14/genericipaddressfield.py 2013-02-21 10:28:40 +0000
@@ -0,0 +1,50 @@
+# flake8: noqa
+# Extract of Django 1.4's db/models/fields/__init__.py file with modified
+# imports.
+from django.core import exceptions
+from django.db.models.fields import Field
+from django.utils.translation import ugettext_lazy as _
+from maasserver.dj14.forms import GenericIPAddressFormField
+from maasserver.dj14.ipv6 import clean_ipv6_address
+from maasserver.dj14.validators import ip_address_validators
+
+
+class GenericIPAddressField(Field):
+ empty_strings_allowed = True
+ description = _("IP address")
+ default_error_messages = {}
+
+ def __init__(self, protocol='both', unpack_ipv4=False, *args, **kwargs):
+ self.unpack_ipv4 = unpack_ipv4
+ self.default_validators, invalid_error_message = \
+ ip_address_validators(protocol, unpack_ipv4)
+ self.default_error_messages['invalid'] = invalid_error_message
+ kwargs['max_length'] = 39
+ Field.__init__(self, *args, **kwargs)
+
+ def get_internal_type(self):
+ return "GenericIPAddressField"
+
+ def to_python(self, value):
+ if value and ':' in value:
+ return clean_ipv6_address(value,
+ self.unpack_ipv4, self.error_messages['invalid'])
+ return value
+
+ def get_db_prep_value(self, value, connection, prepared=False):
+ if not prepared:
+ value = self.get_prep_value(value)
+ return value or None
+
+ def get_prep_value(self, value):
+ if value and ':' in value:
+ try:
+ return clean_ipv6_address(value, self.unpack_ipv4)
+ except exceptions.ValidationError:
+ pass
+ return value
+
+ def formfield(self, **kwargs):
+ defaults = {'form_class': GenericIPAddressFormField}
+ defaults.update(kwargs)
+ return super(GenericIPAddressField, self).formfield(**defaults)
=== added file 'src/maasserver/dj14/ipv6.py'
--- src/maasserver/dj14/ipv6.py 1970-01-01 00:00:00 +0000
+++ src/maasserver/dj14/ipv6.py 2013-02-21 10:28:40 +0000
@@ -0,0 +1,270 @@
+# flake8: noqa
+# Copy of Django 1.4's django/utils/ipv6.py file.
+# This code was mostly based on ipaddr-py
+# Copyright 2007 Google Inc. http://code.google.com/p/ipaddr-py/
+# Licensed under the Apache License, Version 2.0 (the "License").
+from django.core.exceptions import ValidationError
+
+
+def clean_ipv6_address(ip_str, unpack_ipv4=False,
+ error_message="This is not a valid IPv6 address"):
+ """
+ Cleans a IPv6 address string.
+
+ Validity is checked by calling is_valid_ipv6_address() - if an
+ invalid address is passed, ValidationError is raised.
+
+ Replaces the longest continious zero-sequence with "::" and
+ removes leading zeroes and makes sure all hextets are lowercase.
+
+ Args:
+ ip_str: A valid IPv6 address.
+ unpack_ipv4: if an IPv4-mapped address is found,
+ return the plain IPv4 address (default=False).
+ error_message: A error message for in the ValidationError.
+
+ Returns:
+ A compressed IPv6 address, or the same value
+
+ """
+ best_doublecolon_start = -1
+ best_doublecolon_len = 0
+ doublecolon_start = -1
+ doublecolon_len = 0
+
+ if not is_valid_ipv6_address(ip_str):
+ raise ValidationError(error_message)
+
+ # This algorithm can only handle fully exploded
+ # IP strings
+ ip_str = _explode_shorthand_ip_string(ip_str)
+
+ ip_str = _sanitize_ipv4_mapping(ip_str)
+
+ # If needed, unpack the IPv4 and return straight away
+ # - no need in running the rest of the algorithm
+ if unpack_ipv4:
+ ipv4_unpacked = _unpack_ipv4(ip_str)
+
+ if ipv4_unpacked:
+ return ipv4_unpacked
+
+ hextets = ip_str.split(":")
+
+ for index in range(len(hextets)):
+ # Remove leading zeroes
+ hextets[index] = hextets[index].lstrip('0')
+ if not hextets[index]:
+ hextets[index] = '0'
+
+ # Determine best hextet to compress
+ if hextets[index] == '0':
+ doublecolon_len += 1
+ if doublecolon_start == -1:
+ # Start of a sequence of zeros.
+ doublecolon_start = index
+ if doublecolon_len > best_doublecolon_len:
+ # This is the longest sequence of zeros so far.
+ best_doublecolon_len = doublecolon_len
+ best_doublecolon_start = doublecolon_start
+ else:
+ doublecolon_len = 0
+ doublecolon_start = -1
+
+ # Compress the most suitable hextet
+ if best_doublecolon_len > 1:
+ best_doublecolon_end = (best_doublecolon_start +
+ best_doublecolon_len)
+ # For zeros at the end of the address.
+ if best_doublecolon_end == len(hextets):
+ hextets += ['']
+ hextets[best_doublecolon_start:best_doublecolon_end] = ['']
+ # For zeros at the beginning of the address.
+ if best_doublecolon_start == 0:
+ hextets = [''] + hextets
+
+ result = ":".join(hextets)
+
+ return result.lower()
+
+
+def _sanitize_ipv4_mapping(ip_str):
+ """
+ Sanitize IPv4 mapping in a expanded IPv6 address.
+
+ This converts ::ffff:0a0a:0a0a to ::ffff:10.10.10.10.
+ If there is nothing to sanitize, returns an unchanged
+ string.
+
+ Args:
+ ip_str: A string, the expanded IPv6 address.
+
+ Returns:
+ The sanitized output string, if applicable.
+ """
+ if not ip_str.lower().startswith('0000:0000:0000:0000:0000:ffff:'):
+ # not an ipv4 mapping
+ return ip_str
+
+ hextets = ip_str.split(':')
+
+ if '.' in hextets[-1]:
+ # already sanitized
+ return ip_str
+
+ ipv4_address = "%d.%d.%d.%d" % (
+ int(hextets[6][0:2], 16),
+ int(hextets[6][2:4], 16),
+ int(hextets[7][0:2], 16),
+ int(hextets[7][2:4], 16),
+ )
+
+ result = ':'.join(hextets[0:6])
+ result += ':' + ipv4_address
+
+ return result
+
+def _unpack_ipv4(ip_str):
+ """
+ Unpack an IPv4 address that was mapped in a compressed IPv6 address.
+
+ This converts 0000:0000:0000:0000:0000:ffff:10.10.10.10 to 10.10.10.10.
+ If there is nothing to sanitize, returns None.
+
+ Args:
+ ip_str: A string, the expanded IPv6 address.
+
+ Returns:
+ The unpacked IPv4 address, or None if there was nothing to unpack.
+ """
+ if not ip_str.lower().startswith('0000:0000:0000:0000:0000:ffff:'):
+ return None
+
+ hextets = ip_str.split(':')
+ return hextets[-1]
+
+def is_valid_ipv6_address(ip_str):
+ """
+ Ensure we have a valid IPv6 address.
+
+ Args:
+ ip_str: A string, the IPv6 address.
+
+ Returns:
+ A boolean, True if this is a valid IPv6 address.
+
+ """
+ from django.core.validators import validate_ipv4_address
+
+ # We need to have at least one ':'.
+ if ':' not in ip_str:
+ return False
+
+ # We can only have one '::' shortener.
+ if ip_str.count('::') > 1:
+ return False
+
+ # '::' should be encompassed by start, digits or end.
+ if ':::' in ip_str:
+ return False
+
+ # A single colon can neither start nor end an address.
+ if ((ip_str.startswith(':') and not ip_str.startswith('::')) or
+ (ip_str.endswith(':') and not ip_str.endswith('::'))):
+ return False
+
+ # We can never have more than 7 ':' (1::2:3:4:5:6:7:8 is invalid)
+ if ip_str.count(':') > 7:
+ return False
+
+ # If we have no concatenation, we need to have 8 fields with 7 ':'.
+ if '::' not in ip_str and ip_str.count(':') != 7:
+ # We might have an IPv4 mapped address.
+ if ip_str.count('.') != 3:
+ return False
+
+ ip_str = _explode_shorthand_ip_string(ip_str)
+
+ # Now that we have that all squared away, let's check that each of the
+ # hextets are between 0x0 and 0xFFFF.
+ for hextet in ip_str.split(':'):
+ if hextet.count('.') == 3:
+ # If we have an IPv4 mapped address, the IPv4 portion has to
+ # be at the end of the IPv6 portion.
+ if not ip_str.split(':')[-1] == hextet:
+ return False
+ try:
+ validate_ipv4_address(hextet)
+ except ValidationError:
+ return False
+ else:
+ try:
+ # a value error here means that we got a bad hextet,
+ # something like 0xzzzz
+ if int(hextet, 16) < 0x0 or int(hextet, 16) > 0xFFFF:
+ return False
+ except ValueError:
+ return False
+ return True
+
+
+def _explode_shorthand_ip_string(ip_str):
+ """
+ Expand a shortened IPv6 address.
+
+ Args:
+ ip_str: A string, the IPv6 address.
+
+ Returns:
+ A string, the expanded IPv6 address.
+
+ """
+ if not _is_shorthand_ip(ip_str):
+ # We've already got a longhand ip_str.
+ return ip_str
+
+ new_ip = []
+ hextet = ip_str.split('::')
+
+ # If there is a ::, we need to expand it with zeroes
+ # to get to 8 hextets - unless there is a dot in the last hextet,
+ # meaning we're doing v4-mapping
+ if '.' in ip_str.split(':')[-1]:
+ fill_to = 7
+ else:
+ fill_to = 8
+
+ if len(hextet) > 1:
+ sep = len(hextet[0].split(':')) + len(hextet[1].split(':'))
+ new_ip = hextet[0].split(':')
+
+ for _ in xrange(fill_to - sep):
+ new_ip.append('0000')
+ new_ip += hextet[1].split(':')
+
+ else:
+ new_ip = ip_str.split(':')
+
+ # Now need to make sure every hextet is 4 lower case characters.
+ # If a hextet is < 4 characters, we've got missing leading 0's.
+ ret_ip = []
+ for hextet in new_ip:
+ ret_ip.append(('0' * (4 - len(hextet)) + hextet).lower())
+ return ':'.join(ret_ip)
+
+
+def _is_shorthand_ip(ip_str):
+ """Determine if the address is shortened.
+
+ Args:
+ ip_str: A string, the IPv6 address.
+
+ Returns:
+ A boolean, True if the address is shortened.
+
+ """
+ if ip_str.count('::') == 1:
+ return True
+ if filter(lambda x: len(x) < 4, ip_str.split(':')):
+ return True
+ return False
=== added file 'src/maasserver/dj14/validators.py'
--- src/maasserver/dj14/validators.py 1970-01-01 00:00:00 +0000
+++ src/maasserver/dj14/validators.py 2013-02-21 10:28:40 +0000
@@ -0,0 +1,47 @@
+# flake8: noqa
+# Extract of Django 1.4's core/validators.py file with modified imports.
+import re
+
+from django.core.exceptions import ValidationError
+from django.core.validators import RegexValidator
+from django.utils.translation import ugettext_lazy as _
+from maasserver.dj14.ipv6 import is_valid_ipv6_address
+
+
+ipv4_re = re.compile(r'^(25[0-5]|2[0-4]\d|[0-1]?\d?\d)(\.(25[0-5]|2[0-4]\d|[0-1]?\d?\d)){3}$')
+validate_ipv4_address = RegexValidator(ipv4_re, _(u'Enter a valid IPv4 address.'), 'invalid')
+
+def validate_ipv6_address(value):
+ if not is_valid_ipv6_address(value):
+ raise ValidationError(_(u'Enter a valid IPv6 address.'), code='invalid')
+
+def validate_ipv46_address(value):
+ try:
+ validate_ipv4_address(value)
+ except ValidationError:
+ try:
+ validate_ipv6_address(value)
+ except ValidationError:
+ raise ValidationError(_(u'Enter a valid IPv4 or IPv6 address.'), code='invalid')
+
+ip_address_validator_map = {
+ 'both': ([validate_ipv46_address], _('Enter a valid IPv4 or IPv6 address.')),
+ 'ipv4': ([validate_ipv4_address], _('Enter a valid IPv4 address.')),
+ 'ipv6': ([validate_ipv6_address], _('Enter a valid IPv6 address.')),
+}
+
+def ip_address_validators(protocol, unpack_ipv4):
+ """
+ Depending on the given parameters returns the appropriate validators for
+ the GenericIPAddressField.
+
+ This code is here, because it is exactly the same for the model and the form field.
+ """
+ if protocol != 'both' and unpack_ipv4:
+ raise ValueError(
+ "You can only use `unpack_ipv4` if `protocol` is set to 'both'")
+ try:
+ return ip_address_validator_map[protocol.lower()]
+ except KeyError:
+ raise ValueError("The protocol '%s' is unknown. Supported: %s"
+ % (protocol, ip_address_validator_map.keys()))
=== modified file 'src/maasserver/models/__init__.py'
--- src/maasserver/models/__init__.py 2012-09-28 15:32:06 +0000
+++ src/maasserver/models/__init__.py 2013-02-21 10:28:40 +0000
@@ -26,6 +26,18 @@
'UserProfile',
]
+# If we're running Django 1.3, monkey patch Django to create the
+# GenericIPAddressField that was introduced in Django 1.4.
+from django import VERSION
+
+
+if VERSION[0:2] == (1, 3):
+ import django.db.models.fields as django_fields
+ import django.db.models as django_models
+ from maasserver.dj14.genericipaddressfield import GenericIPAddressField
+ django_fields.GenericIPAddressField = GenericIPAddressField
+ django_models.GenericIPAddressField = GenericIPAddressField
+
from logging import getLogger
from django.contrib import admin
