Exclude filter for display doesn't work

[rochi]

If I want to display only open tickets with

display!=status:closed

The closed tickets are still displayed.

[ChrisNelson]

Replying to rochi:

If I want to display only open tickets with

display!=status:closed

The closed tickets are still displayed.

The filter specifies what tickets to include. The way it is written there really isn't a way to negate that to exclude tickets. We might add an "hide" filter (the opposite of display) so you could

hide=status:closed

[rochi]

In your code comments you are talking about TracQuery so I was a little bit confused.
That would be a variant.

But, I've also tried:

display=status:new|status:assigned|status:accepted

It shows also not all posible tickets.

[rochi]

I've done some changes for the filter algorithm, it's not perfect and there are a lof of TODOs...

# Process each part into the display filter
                displayFilter = {}
                for f in displayList:
                    field, value = f.split(':')
                    if field in displayFilter:
                        displayFilter[field].append(value)
                    else:
                        displayFilter[field] = [value]

                # Filter the tickets
                displayTickets = []
                self.env.log.debug("Tickets available: %s",
                           len(self.tickets))
                for ticket in self.tickets:
                    # Default to showing every ticket
                    display = True
                    # Process each element and disable display if all
                    # filters fails to match. ((or) disjunction)
                    for f in displayFilter:
                        for v in displayFilter[f]:
                            if ticket[f] == v:
                                #self.env.log.debug("Display filter match '%s': '%s' for ticket #%s",
                                 #   f,
                                 #   v,
                                 #   ticket['id'])
                                display = True
                                break
                            #self.env.log.debug("Display filter '%s' not match: '%s' for ticket #%s",
                            #    f,
                            #    v,
                            #    ticket['id'])
                            display = False

                    if display:
                        displayTickets.append(ticket)

                # Sort the tickets
                displayTickets.sort(self._compare_tickets)

Further work:

• disjunction for fields with same key otherwise conjunction.

There is still a problem with the order of subtasks, they are sometimes placed far away from their parents.

[ChrisNelson]

Replying to rochi:

I've done some changes for the filter algorithm, it's not perfect and there are a lof of TODOs...

Thanks for working on this.

                     if field in displayFilter:
                         displayFilter[field].append(value)
                     else:
                         displayFilter[field] = [value]

I wonder if value shouldn't be split on |. You can do id=1|2|3, so I think it would be natural to do display=owner:curly|moe.

Further work:

• disjunction for fields with same key otherwise conjunction.

There is still a problem with the order of subtasks, they are sometimes placed far away from their parents.

That may be deliberate. My original use case for filters was: schedule everything then show me what I should do next" so I sort by date after filtering. You could add a displaySort option to control that.

[rochi]

Replying to ChrisNelson:

I wonder if value shouldn't be split on |. You can do id=1|2|3, so I think it would be natural to do display=owner:curly|moe.

You are right, but I think it's better to support both.

That may be deliberate. My original use case for filters was: schedule everything then show me what I should do next" so I sort by date after filtering. You could add a displaySort option to control that.

You missed my point, move the next task to the top is the correct behavior, but on a parent-child relation the parent have also move up.
This can be a variant:

• first item to schedule
  • child from parent next to schedule
• parent
  • other childs

[rochi]

Support for: disjunction for fields with same key otherwise conjunction

                # Process each part into the display filter
                displayFilter = {}
                for f in displayList:
                    field, value = f.split(':')
                    if field in displayFilter:
                        displayFilter[field].append(value)
                    else:
                        displayFilter[field] = [value]

                # Filter the tickets
                displayTickets = []
                self.env.log.debug("Tickets available: %s",
                           len(self.tickets))
                for ticket in self.tickets:
                    # Default to showing every ticket
                    fieldDisplay = True
                    # Process each element and disable display if all
                    # filters fails to match. ((or) disjunction)
                    for f in displayFilter:
                        display = True
                        for v in displayFilter[f]:
                            if ticket[f] == v:
                                #self.env.log.debug("Display filter match '%s': '%s' for ticket #%s",
                                 #   f,
                                 #   v,
                                 #   ticket['id'])
                                display = True
                                break
                            #self.env.log.debug("Display filter '%s' not match: '%s' for ticket #%s",
                            #    f,
                            #    v,
                            #    ticket['id'])
                            display = False
                        fieldDisplay = fieldDisplay & display
                    if fieldDisplay:
                        displayTickets.append(ticket)

                # Sort the tickets
                displayTickets.sort(self._compare_tickets)

[rochi]

Ok, I've finished my improvements (see patch)

New features are:

• Filter with 'display'
  • display:status=new|status=assigned or by status=new|assigned
• The parent is now always on top of its childs and still considereds the schedule
  • The first item in schedule is than the second item on front because the parent is always the first item

I'm sorry for some code snippets looking strange, but I'm very new to python.

[rochi]

See comment:7

[ChrisNelson]

Replying to rochi:

Ok, I've finished my improvements (see patch)

Thanks! I'll try to review it in the next few days.

New features are:

• Filter with 'display'
  • display:status=new|status=assigned or by status=new|assigned
• The parent is now always on top of its childs and still considereds the schedule
  • The first item in schedule is than the second item on front because the parent is always the first item

I'm sorry for some code snippets looking strange, but I'm very new to python.

No problem. If the problems are minor, I'll clean them up. If I don't understand something, I'll let you know.

[ChrisNelson]

I tried to apply this patch and can't get it to work. I end up with:

            # If no display filter, just display all tickets
            if not options.get('display') or options['display'] == '':
                displayTickets = self.tickets
            # Otherwise, process the filter
            else:
                # Build the list of display filters from the configured value
                # The general form is
                # 'display=field:value|field:value...'. Split on pipe
                # to get each part
                displayList = options['display'].split('|')

                # Process each part into the display filter
                displayFilter = {}
                for f in displayList:
                    field, value = f.split(':')
                    values = value.split('|')
                    if field in displayFilter:
                        displayFilter[field].extend(values)
                    else:
                        displayFilter[field] = values

Note that the display filter is parsed twice on '|': once for multiple fields (field1:value1|field2:value2) and once for multiple values (field1:value1|value2).

I imagine both could be supported if we kept track of the previous field so that if the second split failed, we'd use the first field. That is, field1:value1|value2 would parse to field1:value1 and value2 and when the split of value2 on : failed, the code would know to look back to field1. This is more than I can do right now.

[rochi]

That ist only the processing of the filters, right?
The real ticket filtering looks like:

# Filter the tickets
                displayTickets = []
                self.env.log.debug("Tickets available: %s",
                           len(self.tickets))
                self.ticketsByID.clear()
                for ticket in self.tickets:
                    # Default to showing every ticket
                    fieldDisplay = True
                    # Process each element and disable display if all
                    # filters fails to match. ((or) disjunction)
                    for f in displayFilter:
                        display = True
                        for v in displayFilter[f]:
                            if ticket[f] == v:
                                #self.env.log.debug("Display filter match '%s': '%s' for ticket #%s",
                                #   f,
                                #   v,
                                #   ticket['id'])
                                display = True
                                break
                            #self.env.log.debug("Display filter '%s' not match: '%s' for ticket #%s",
                            #    f,
                            #    v,
                            #    ticket['id'])
                            display = False
                        fieldDisplay = fieldDisplay & display
                    if fieldDisplay:
                        displayTickets.append(ticket)
                        self.ticketsByID[ticket['id']] = ticket

                # Sort the tickets by date and successor dependencies
                displayTickets.sort(self._compare_tickets)

and comes directly after the filter processing.

What is yout specific issue?
I have this code now running several months without problems.

[anonymous]

Replying to rochi:

That ist only the processing of the filters, right?
...
What is yout specific issue?
I have this code now running several months without problems.

I have have a chart that shows 7 tasks if I don't add a display option.

• If I use display=owner:owner1|owner:owner2 I get "No tasks selected".
• If I use display=owner:owner1|owner2, I get "need more than 1 value to unpack" from field, value = f.split(':')

[ChrisNelson]

(In [11704]) Allow display=field:val1|val2... Refs #9983.

[anonymous]

After [11704], I can do

• display=field1:value1|field2:value2
• display=field1:value1|field1:value2
• display=field1:value1|value2

and all work as expected.

[ChrisNelson]

Balance of previous patch which implements sorting

[ChrisNelson]

I'm not yet comfortable enough with the sorting additions to put them on the main line but I attached a patch which should implement them on top of my commit.