id,summary,reporter,owner,description,type,status,priority,component,severity,resolution,keywords,cc,release
639,Large p4 depots kill virtual memory (patch included),r.blum@gmx.net,lewisbaker,If you do an initial sync to a large depot (50_000 changelists)_ the perforce plugin will use up VM like crazy. (I stopped once I reached about 1.5 GB) \r\n\r\nSmall modification to the sync procedure solves that problem. (Basically_ get changelists in chunks of 1000). Here's my local mod\r\n\r\n{{{\r\n#!python\r\n   # Override sync to precache data to make it run faster\r\n    def sync(self):\r\n        youngest_stored = self.repos.get_youngest_rev_in_cache(self.db)     \r\n        if youngest_stored is None:\r\n            youngest_stored = '0'\r\n\r\n        while youngest_stored != str(self.repos.youngest_rev):\r\n            # Need to cache all information for changes since the last\r\n            # sync operation.\r\n\r\n            youngest_to_get = self.repos.youngest_rev\r\n            if youngest_to_get > int(youngest_stored) + 1000:\r\n                youngest_to_get = int(youngest_stored) + 1000\r\n\r\n            # Obtain a list of changes since the last cache sync\r\n            from p4trac.repos import _P4ChangesOutputConsumer\r\n            output = _P4ChangesOutputConsumer(self.repos._repos)\r\n            self.repos._connection.run('changes'_ '-l'_ '-s'_ 'submitted'_\r\n                                       '@>%s_%d' % ( youngest_stored_ youngest_to_get )_\r\n                                       output=output)\r\n\r\n            if output.errors:\r\n                from p4trac.repos import PerforceError\r\n                raise PerforceError(output.errors)\r\n\r\n            changes = output.changes\r\n            changes.reverse()\r\n\r\n            # Perform the precaching of the file history for files in these\r\n            # changes.\r\n            self.repos._repos.precacheFileHistoryForChanges(changes)\r\n\r\n            youngest_stored=str(youngest_to_get)\r\n\r\n        # Call on to the default implementation now that we've cached\r\n        # enough information to make it run a bit faster.\r\n        CachedRepository.sync(self)\r\n}}},enhancement,closed,normal,PerforcePlugin,normal,duplicate,needinfo,,0.10