Modify

Opened 6 years ago

Closed 6 years ago

Last modified 5 years ago

#3466 closed defect (invalid)

No matches found

Reported by: anonymous Owned by: airadier
Priority: normal Component: SearchAllPlugin
Severity: normal Keywords:
Cc: Trac Release: 0.10

Description

Hello,

I've just install the tracsearchall plugin, and have the All projects checkbox.

When I want to search something, each time I get the No matches found message... It seem that I've no (python) error messages.

Attachments (3)

trac.log (2.2 KB) - added by anonymous 6 years ago.
searchall.py (4.6 KB) - added by bobbysmith007 6 years ago.
a new version of the search all file that contains repository search as well
searchall.2.py (5.1 KB) - added by bobbysmith007 6 years ago.
A version that searches the repo and identifies files in the repository that match a given regex

Download all attachments as: .zip

Change History (33)

comment:1 follow-up: Changed 6 years ago by bobbysmith007

Are you sure there were supposed to be any matches?

Can you post a relevant section from your TracLog? (set it to debug)

comment:2 in reply to: ↑ 1 Changed 6 years ago by anonymous

Replying to bobbysmith007:

Are you sure there were supposed to be any matches?

Can you post a relevant section from your TracLog? (set it to debug)

Thanks for your reply !

Yes, I'm sure that I use a string that must matche with some project files.

I've send the log file as attachment.

Changed 6 years ago by anonymous

comment:3 follow-up: Changed 6 years ago by bobbysmith007

Are you sure that the plugin is enabled? I see the egg loading (not sure if that means its enabled), but it should be logging what evironments it is searching in. Do you have the 'All projects' checkbox? Is it Checked? (not trying to be a pain, just want to check all of the obvious stuff)

In my setup, where it works, we have a directory /var/trac that contains all our separate instances of trac. When it is searching, it just goes up one dir from the current location and then starts going to town treating each subdir as a trac environment and querying it. Is there something different in your setup from what I have described?

comment:4 follow-up: Changed 6 years ago by bobbysmith007

comment:5 in reply to: ↑ 3 Changed 6 years ago by anonymous

Replying to bobbysmith007:

Are you sure that the plugin is enabled? I see the egg loading (not sure if that means its enabled), but it should be logging what evironments it is searching in. Do you have the 'All projects' checkbox? Is it Checked?

Yes, I've the All projects checkbox and checked. I've edit the trac.ini to enable the plugin.

Replying to bobbysmith007:

(not trying to be a pain, just want to check all of the obvious stuff)

No problems ;) I'm IT and it's like that I'm working :)

Replying to bobbysmith007:

In my setup, where it works, we have a directory /var/trac that contains all our separate instances of trac. When it is searching, it just goes up one dir from the current location and then starts going to town treating each subdir as a trac environment and querying it. Is there something different in your setup from what I have described?

Yes. I've create /home/roottrac/projects/, and the projects/ folder contain all of our projects.

comment:6 Changed 6 years ago by bobbysmith007

Well its not my plugin, so I dont know how they would want to rectify this upstream.

That said, if you just want it to work, you might try just adding a line after line 41 of the search file that explicitly sets the search_path variable to be the /home/roottrac/projects folder.

      search_path, this_project = os.path.split(self.env.path)
      search_path = '/home/roottrac/projects'

HTH,

Russ

comment:7 Changed 6 years ago by anonymous

I've modify the script, but that don't work... :(

comment:8 Changed 6 years ago by bobbysmith007

What are the results now? Are you getting an error in the trac log, or getting exactly the same results? I would have expected that to take care of it. You might try adding some more logging around that function and see if you can identify where its going wrong. I might suggest logging each directory you are trying to search (line 51) before it can decided it doesnt want to search that dir. Is this list of directories the correct trac environments? If so, perhaps one of the other tests is causing it to fail.

comment:9 Changed 6 years ago by anonymous

The result is the same.

Ok, it was my idea, to add logs, but I don't know Python ( at this time ), and I've quick search on the net, but not found ( search during 2 min because, back home just after ;) )

So, I will learn python and I'll see what happen.

comment:10 in reply to: ↑ 4 Changed 6 years ago by bobbysmith007

All you should really need to know to add the logging is that indention matters and that a log message looks like (and I guess also string formatting):

http://trac-hacks.org/browser/searchallplugin/tracsearchall/searchall.py#L65

I would suggest something like:

#line 49
self.env.log.debug("Searching directory %s" % search_path)
for project in os.listdir(search_path):
    self.env.log.debug("Searching project %s from %s" % (os.path.join(search_path,project), this_project))
    if project == this_project:
        continue

Might I also recommend Dive Into Python. I found this book to be pretty useful when learning python.

comment:11 Changed 6 years ago by anonymous

Thanks for your help !

I've copy past after line 49 your code and trac made an Trac detected an internal error.

This is Python Traceback :

Traceback (most recent call last):
  File "/var/lib/python-support/python2.5/trac/web/main.py", line 406, in dispatch_request
    dispatcher.dispatch(req)
  File "/var/lib/python-support/python2.5/trac/web/main.py", line 237, in dispatch
    resp = chosen_handler.process_request(req)
  File "/var/lib/python-support/python2.5/trac/Search.py", line 181, in process_request
    results += list(source.get_search_results(req, terms, filters))
  File "build/bdist.linux-i686/egg/tracsearchall/searchall.py", line 63, in get_search_results
  File "/var/lib/python-support/python2.5/trac/env.py", line 462, in open_environment
    env = Environment(env_path)
  File "/var/lib/python-support/python2.5/trac/env.py", line 142, in __init__
    load_components(self)
  File "/var/lib/python-support/python2.5/trac/loader.py", line 90, in load_components
    for name in egg.get_entry_map('trac.plugins'):
  File "/usr/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/pkg_resources.py", line 2187, in get_entry_map
    self._get_metadata('entry_points.txt'), self
  File "/usr/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/pkg_resources.py", line 1992, in parse_map
    for group, lines in data:
  File "/usr/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/pkg_resources.py", line 2518, in split_sections
    for line in yield_lines(s):
  File "/usr/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/pkg_resources.py", line 1813, in yield_lines
    for ss in strs:
  File "/usr/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/pkg_resources.py", line 2121, in _get_metadata
    for line in self.get_metadata_lines(name):
  File "/usr/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/pkg_resources.py", line 1140, in get_metadata_lines
    return yield_lines(self.get_metadata(name))
  File "/usr/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/pkg_resources.py", line 1137, in get_metadata
    return self._get(self._fn(self.egg_info,name))
  File "/usr/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg/pkg_resources.py", line 1195, in _get
    return self.loader.get_data(path)
ZipImportError: bad local file header in /usr/lib/python2.5/site-packages/tracsearchall-0.2-py2.5.egg

comment:12 Changed 6 years ago by anonymous

Ok, it's ok. I've just restart apache and that work now. (should be a egg cached).

So, now, my debug messages :

2008-08-19 16:49:49,180 Trac[searchall] DEBUG: Searching directory /home/roottrac
2008-08-19 16:49:49,181 Trac[searchall] DEBUG: Searching project /home/roottrac/index.html from projects
2008-08-19 16:49:49,181 Trac[searchall] DEBUG: Searching project /home/roottrac/projects from projects

comment:13 Changed 6 years ago by anonymous

I just want to explain you something because, after read your code and the result, I see maybe the issue.

We are using, currently, VSS. Our goal is to switch to SVN. So, I've use a script to import our VSS database to SVN. So, ALL of our projects are in the /projects folder.

The /project folder contain this files: attachments conf db htdocs log plugins README templates VERSION wiki-macros

I think that the plugin search for projects folders ( One folder for one project ). But, I've just one folder with ALL projects. ( It's a easy way for me to import VSS -> SVN ).

comment:14 Changed 6 years ago by bobbysmith007

  • Resolution set to invalid
  • Status changed from new to closed

Ok well if you have all of your projects imported into a single trac, then this plugin will not help you (it is about searching many tracs at once, which is what I thought you were saying).

The /projects folder above contains a single trac instance (not a SVN repo, which you must have elsewhere).

If what you are trying to do is search your source control repo, then you will need another plugin / program that can do that. Trac will only search the SVN commit messages by default, not all of the code that is checked in.

HTH, closing this ticket as invalid. If you have any further questions feel free to ask.

Russ

comment:15 Changed 6 years ago by anonymous

Okay,

Thank you bobbysmith007 for all of your help !

I was thinking that as trac have the browse source feature, it should have plugin to search in it... Maybe you can suggest me another method to import VSS in SVN, to permit to the searchall plugin to work fine ? ( Just to inform you: The script to export data from VSS extract all if I specify the / path, but if I'll give it each projects path, I will have one extract file by projects, and so could create one folder by project. )

comment:16 Changed 6 years ago by anonymous

In fact, the SearchAllPlugin don't search in source code, but in Wiki, tickets, and Changesets in all projects ?

If it's the case, is exist a way to search through Trac in source code a string ?

comment:17 Changed 6 years ago by bobbysmith007

Unfortunately trac does not support searching through a repository for a source code string. There are a couple of projects (not trac) that attempt to support this, the svn api/client doesn't support direct searching of a repo (apparently), only asking for a version (which someone is free to seach).

Keep an eye on this ticket as it seems like it might be reasonable to do those searches if you are willing to wait (an svn repo can get pretty huge, esp if you dump all projects into one). If I hack something together I will notify you by updating this ticket.

comment:18 Changed 6 years ago by anonymous

Thank you so much !

I'm so happy to read that ! :)

comment:19 follow-up: Changed 6 years ago by bobbysmith007

Alright, I cannot claim that this will in anyway be a quick process, but I have a search option that will allow you (and us incidentally) to search the svn repository. This may take a LONG time if your repo is big (like tens of minutes to complete), but if thats what you need to do then at least it is there.

This isnt my plugin so I cannot commit this change. I might go ahead and make this a separate plugin, but for right now you can try it out by dropping this new version of the searchall.py in place and then reinstalling (attachment coming up next)

Changed 6 years ago by bobbysmith007

a new version of the search all file that contains repository search as well

comment:20 in reply to: ↑ 19 Changed 6 years ago by anonymous

Whaooo !! Thanks mr bobbysmith007 !!!

Replying to bobbysmith007:

This may take a LONG time if your repo is big (like tens of minutes to complete), but if thats what you need to do then at least it is there.

Is it because you browse the history ?

I'll try and tell you ;)

comment:21 Changed 6 years ago by anonymous

I've try your modification. So, now it search in repository, but return me as result all modifications commited. And not my string.

Tell me if I can help you.

comment:22 Changed 6 years ago by bobbysmith007

Basically this patch locates all changesets where some phrase occured (using regular expressions). If your change sets are large I could see this not being much help (because all changesets would contain that word) being a problem. I might be able to add links to the specific files, but that would make the search take even longer (right now I stop searching the changeset the moment I find a copy of a file contains that phrase).

What would you like to see. I am thinking of adding a couple more modes anyway so if you can describe what you suggested I might be able to make that happen.

comment:23 Changed 6 years ago by bobbysmith007

Alright I might have a better solution for you (at the expense of the one I posted earlier).

The following searchall.py will allow you to search the repo for files that contain the specified regex at the most recent version that they contain it.

Changed 6 years ago by bobbysmith007

A version that searches the repo and identifies files in the repository that match a given regex

comment:24 Changed 6 years ago by anonymous

Okay, you've made exactly what I would ask you ! :)

Search in the HEAD version. I've install your version, and it work fine ! Take a lot of time ... but it's better ! ( My repository size is )

I've little idea :

  • Don't check binary files (.exe, .doc, .xls, .zip, etc ...)
  • Don't check orphaned files
  • Have the possibility to set a path where search.
  • Have the possibility to search the string in a specified filename.

( This idea should be crazy, but is not. I've a script written as template that I use to make a new script. So, many projects have some files with the same name, but the code is different. So I need to search the string 'test' in file outscript.bas for example )

Thanks again for all of your help !!

comment:25 Changed 6 years ago by anonymous

Sorry: My repository size is 1.4GB.

Before, with VSS, it was 4GB !!

comment:26 Changed 6 years ago by anonymous

Just to inform you, I've switch to trac 0.11 yesterday.

comment:27 Changed 6 years ago by anonymous

Hello. Are you back ? :)

comment:28 follow-up: Changed 6 years ago by bobbysmith007

Just FYI, I am not going to add this stuff. There is probably enough there that you could add it if you tried, but I really dont have the time at the moment.

Sorry

comment:29 in reply to: ↑ 28 Changed 6 years ago by anonymous

Replying to bobbysmith007:

Just FYI, I am not going to add this stuff. There is probably enough there that you could add it if you tried, but I really dont have the time at the moment.

Sorry

Okay, no problemo. I will try ;)

comment:30 Changed 5 years ago by andreavb1985@…

Thanks, guys,

I have a '/home/trac/' directory containing all my Trac projects (instead of a Master Project) and this ticket was very useful to me, specially comment 6.
In about 5 minutes I had installed and configured SearchAllPlugin, and it was already working! As I have 300+ projects, it's slowly, but... it works, and it's very very very useful!!

Add Comment

Modify Ticket

Action
as closed The owner will remain airadier.
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.