← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #27117

[Bug 1413619] [NEW] when container sync run, already deleted object is synced

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

I set container  to sync container1 to container2.

Then, In container1 uploaded serveral files.
After container sync run,   all files of container 1  was copied to container2.
This is no problem. 

And then, I  deleted all files of container1.
After container sync run,   all files of container2 was deleted.
and Also this no problem.

But since this,  whenever container sync run, In container_sync.log next
error printed.

Jan 22 22:57:26 dev-swift02 container-sync: ERROR Syncing /data1/sdc/containers/46110/8b8/b41eda9ca916f6262b8986fae95648b8/b41eda9ca916f6262b8986fae95648b8.db {'name': 'file1', 'deleted': 1, 'created_at': '1421928707.61045', 'storage_policy_index': 0, 'etag': 'noetag', 'content_type': 'application/deleted', 'ROWID': 729, 'size': 0}: #012Traceback (most recent call last):#012  File "/usr/lib/python2.6/site-packages/swift/container/sync.py", line 362, in container_sync_row#012    logger=self.logger)#012  File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 839, in delete_object#012    client.retry_request('DELETE', **kwargs)#012  File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 805, in retry_request#012    return self.base_request(method, **kwargs)#012  File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 765, in base_request#012    conn = urllib2.urlopen(req, timeout=timeout)#012  File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen#012    return _opener.open(url, data, timeout)#012  File "/usr/lib64/python2.6/urllib2.py", line 397, in open#012    response = meth(req, response)#012  File "/usr/lib64/python2.6/urllib2.py", line 510, in http_response#012    'http', request, response, code, msg, hdrs)#012  File "/usr/lib64/python2.6/urllib2.py", line 435, in error#012    return self._call_chain(*args)#012  File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain#012    result = func(*args)#012  File "/usr/lib64/python2.6/urllib2.py", line 518, in http_error_default#012    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)#012HTTPError: HTTP Error 404: Not Found
Jan 22 22:57:43 dev-swift02 container-sync: ERROR Syncing /data1/sdc/containers/46110/8b8/b41eda9ca916f6262b8986fae95648b8/b41eda9ca916f6262b8986fae95648b8.db {'name': 'file2', 'deleted': 1, 'created_at': '1421928665.53463', 'storage_policy_index': 0, 'etag': 'noetag', 'content_type': 'application/deleted', 'ROWID': 730, 'size': 0}: #012Traceback (most recent call last):#012  File "/usr/lib/python2.6/site-packages/swift/container/sync.py", line 362, in container_sync_row#012    logger=self.logger)#012  File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 839, in delete_object#012    client.retry_request('DELETE', **kwargs)#012  File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 805, in retry_request#012    return self.base_request(method, **kwargs)#012  File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 765, in base_request#012    conn = urllib2.urlopen(req, timeout=timeout)#012  File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen#012    return _opener.open(url, data, timeout)#012  File "/usr/lib64/python2.6/urllib2.py", line 397, in open#012    response = meth(req, response)#012  File "/usr/lib64/python2.6/urllib2.py", line 510, in http_response#012    'http', request, response, code, msg, hdrs)#012  File "/usr/lib64/python2.6/urllib2.py", line 435, in error#012    return self._call_chain(*args)#012  File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain#012    result = func(*args)#012  File "/usr/lib64/python2.6/urllib2.py", line 518, in http_error_default#012    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)#012HTTPError: HTTP Error 404: Not Found

...
...
...


This error cause the file list exist deleted flag 1 only in container meta  db, although the file is deleted already,  
I confirmed meta db of container using sqlite.
sqlite> select * from object;
sqlite> select * from object;
ROWID|name|created_at|size|content_type|etag|deleted|storage_policy_index
727|file1|1421928707.61045|0|application/deleted|noetag|1|0
728|file2|1421928665.53463|0|application/deleted|noetag|1|0
...


When container sync run, the file with deleted flag 1 must skip without sync.
(but at current versin container sync, also the file with deleted flag 1  was sync.)

** Affects: horizon
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1413619

Title:
  when container sync run,  already deleted object is synced

Status in OpenStack Dashboard (Horizon):
  New

Bug description:
  I set container  to sync container1 to container2.

  Then, In container1 uploaded serveral files.
  After container sync run,   all files of container 1  was copied to container2.
  This is no problem. 

  And then, I  deleted all files of container1.
  After container sync run,   all files of container2 was deleted.
  and Also this no problem.

  But since this,  whenever container sync run, In container_sync.log
  next error printed.

  Jan 22 22:57:26 dev-swift02 container-sync: ERROR Syncing /data1/sdc/containers/46110/8b8/b41eda9ca916f6262b8986fae95648b8/b41eda9ca916f6262b8986fae95648b8.db {'name': 'file1', 'deleted': 1, 'created_at': '1421928707.61045', 'storage_policy_index': 0, 'etag': 'noetag', 'content_type': 'application/deleted', 'ROWID': 729, 'size': 0}: #012Traceback (most recent call last):#012  File "/usr/lib/python2.6/site-packages/swift/container/sync.py", line 362, in container_sync_row#012    logger=self.logger)#012  File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 839, in delete_object#012    client.retry_request('DELETE', **kwargs)#012  File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 805, in retry_request#012    return self.base_request(method, **kwargs)#012  File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 765, in base_request#012    conn = urllib2.urlopen(req, timeout=timeout)#012  File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen#012    return _opener.open(url, data, timeout)#012  File "/usr/lib64/python2.6/urllib2.py", line 397, in open#012    response = meth(req, response)#012  File "/usr/lib64/python2.6/urllib2.py", line 510, in http_response#012    'http', request, response, code, msg, hdrs)#012  File "/usr/lib64/python2.6/urllib2.py", line 435, in error#012    return self._call_chain(*args)#012  File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain#012    result = func(*args)#012  File "/usr/lib64/python2.6/urllib2.py", line 518, in http_error_default#012    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)#012HTTPError: HTTP Error 404: Not Found
  Jan 22 22:57:43 dev-swift02 container-sync: ERROR Syncing /data1/sdc/containers/46110/8b8/b41eda9ca916f6262b8986fae95648b8/b41eda9ca916f6262b8986fae95648b8.db {'name': 'file2', 'deleted': 1, 'created_at': '1421928665.53463', 'storage_policy_index': 0, 'etag': 'noetag', 'content_type': 'application/deleted', 'ROWID': 730, 'size': 0}: #012Traceback (most recent call last):#012  File "/usr/lib/python2.6/site-packages/swift/container/sync.py", line 362, in container_sync_row#012    logger=self.logger)#012  File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 839, in delete_object#012    client.retry_request('DELETE', **kwargs)#012  File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 805, in retry_request#012    return self.base_request(method, **kwargs)#012  File "/usr/lib/python2.6/site-packages/swift/common/internal_client.py", line 765, in base_request#012    conn = urllib2.urlopen(req, timeout=timeout)#012  File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen#012    return _opener.open(url, data, timeout)#012  File "/usr/lib64/python2.6/urllib2.py", line 397, in open#012    response = meth(req, response)#012  File "/usr/lib64/python2.6/urllib2.py", line 510, in http_response#012    'http', request, response, code, msg, hdrs)#012  File "/usr/lib64/python2.6/urllib2.py", line 435, in error#012    return self._call_chain(*args)#012  File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain#012    result = func(*args)#012  File "/usr/lib64/python2.6/urllib2.py", line 518, in http_error_default#012    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)#012HTTPError: HTTP Error 404: Not Found

  ...
  ...
  ...

  
  This error cause the file list exist deleted flag 1 only in container meta  db, although the file is deleted already,  
  I confirmed meta db of container using sqlite.
  sqlite> select * from object;
  sqlite> select * from object;
  ROWID|name|created_at|size|content_type|etag|deleted|storage_policy_index
  727|file1|1421928707.61045|0|application/deleted|noetag|1|0
  728|file2|1421928665.53463|0|application/deleted|noetag|1|0
  ...

  
  When container sync run, the file with deleted flag 1 must skip without sync.
  (but at current versin container sync, also the file with deleted flag 1  was sync.)

To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/1413619/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next