Filtering experience

[rragan]

I was trying to isolate just the caches making up a piece of geo art in Map View. The process went like this. 

• Guess about area of art and load caches based on Bounds and center. 
• Scoot over and pick up missed parts
• note that I picked up extraneous nearby caches
• save all to offline list for cleanup
• i wanted only offline caches with Geo-Art in the name. Filter finds these just fine. I wanted a NOT option that would negate the filtered results so I could delete just the non Geo-Art ones
• ok, highlight those visible geo art ones. Then change filters to show non-highlighted ones and delete filtered caches. 

Not so terrible but it could have been done in one step with a NOT. 

With hindsight I should have probably searched the cache name string "Geo-Art" and I would have gotten only the ones I wanted. However, NOT would apply to a variety of filtering not involving title strings. 

[Nic Hubbard]

Do you have the GC code of one of the caches so I can use the same area to replicate this?

[rragan]

Start with GC4QNKH

[Nic Hubbard]

A new Inverse option will be available for all non-boolean filters in the next beta.

Thanks for the suggestion, when building and testing this I realize how powerful it is!

[rragan]

Works great so far. One of the most powerful PC text editors,Textpad, had huge flexibility with only a handful of features

- Mark line/unmark line

- mark lines found by a search plus Mark All/Unmark All

- Invert marks (e.g. NOT)

- Actions on marked lines: delete them, copy/cut them

- Search supports regex 

Except for regex, Cachly now has the primitives: Highlight=Mark,NOT, Filter plus highlight results. Delete filtered results, save visible having filtered on highlight. Highlight ALL is coming I think. 

I am empowered. 

[Nic Hubbard]

4 minutes ago, rragan said:

I am empowered.

Glad you are liking it. Let me know if you run into any issues.

[Bolling]

I just tried a distance filter with inverse as first try. 

Should filters Less than 10 miles and inverse of geater than 10 miles give the same results?  (Thus was tested as one filter at a time). I would have expected so but the inverse greater than filter had results under 10 miles and over 50 miles (or similar value). 

 

Edit two hours later: I can not reproduce. 

Edited January 13, 2018 by Bolling

[Nic Hubbard]

On 1/12/2018 at 5:46 PM, Bolling said:

Edit two hours later: I can not reproduce. 

I wasn't able to either.

[Nic Hubbard]

On 1/12/2018 at 3:03 PM, rragan said:

Except for regex, Cachly now has the primitives: Highlight=Mark,NOT, Filter plus highlight results. Delete filtered results, save visible having filtered on highlight. Highlight ALL is coming I think. 

In beta 11 Filters that use the text type "Matches" now support regex.

[rragan]

Fantastic. 

[rragan]

I'm not seeing where I should input a regex. Hint please 

[Nic Hubbard]

1 hour ago, rragan said:

I'm not seeing where I should input a regex. Hint please 

You will see it when you choose the “Matches” logic type for string filters. 

[rragan]

I tried this on a bunch of files with Geo-Art in the title. My regex was .*Geo and nothing was matched. Changing it to .*Geo.* gave the right matches. Is there an implicit $ on the end or something that prevents the first form matching.

Also I noticed that when entering the match text ,the Done key on the keyboard is not doing anything. I have to tap Done on the filter creation window. I think either should work. 

[Nic Hubbard]

13 hours ago, rragan said:

I tried this on a bunch of files with Geo-Art in the title. My regex was .*Geo and nothing was matched. Changing it to .*Geo.* gave the right matches. Is there an implicit $ on the end or something that prevents the first form matching.

Apple's regex documentation references: http://userguide.icu-project.org/strings/regexp. Does that help to understand the format needed? I will put a reference to this in the app.

13 hours ago, rragan said:

Also I noticed that when entering the match text ,the Done key on the keyboard is not doing anything. I have to tap Done on the filter creation window. I think either should work. 

Will fix!

[rragan]

I appreciate the link as regex features vary a lot from one implementation to the next. Even reading through that, my very simple regex .*Geo should match in the ICU engine and a host of others. I tried it in a tester to be sure I wasn't confused somehow  it matched the title up to and including Geo as I expected. Are you expecting the whole title to match before accepting it as qualifying for the filter?

test text:Asterisk 39 - S*W*A*G Geo-Art

regex: .*Geo

https://regex101.com/r/wG3aM3/1

 

[Nic Hubbard]

2 hours ago, rragan said:

Are you expecting the whole title to match before accepting it as qualifying for the filter?

I am simply passing the regex string to the NSPredicate object which does the matching. Nothing more than that. 

All I can guess is that MATCHES requires the regex to match the entire string, not just part of it. So you have to have .* at the beginning and end to match anywhere in the string. Ref

[rragan]

With that extra info, let me read it again. This seems wrong behavior that will trip people up.

[Nic Hubbard]

25 minutes ago, rragan said:

This seems wrong behavior that will trip people up.

I can't dispute that, but I also don't have any other way to implement it. Testing a 3 line implementation in a brand new project gives the same results, so I guess that is just how regex works on iOS, at least in NSPredicate which is what must be used for database fetches in Core Data.

[rragan]

Parsing this wording carefully

"

• MATCHES: The left hand expression equals the right hand expression using a regex-style comparison according to ICU v3 (for more details see the ICU User Guide for Regular Expressions)."

and all all the examples I can find makes it seem that in this context, anything other than a complete match will fail even if I'm using the regex to see if a particular substring is present which regex users from other languages do all the time. Users will need this explained clearly or they will be puzzled as I was. 

I thought some about prepending .* to the start and end of the user regex. Since this matches zero or more, it might not cause a problem except if the regex has ^ and $ usage. It would magically make lots of things just work but regex's are so complex, I'm not sure there aren't edge cases this would mess up. 

[barefootguru]

Yeah, Google (actually DuckDuckGo) confirms that NSPredicate has to match the entire string.  I do think this will trip people up, as no ^ $ is present.

I like @rragan‘s idea of stripping ^ $ if present and enclosing the string in .* but maybe that’s too complicated, and an explanation on the screen saying the expression is automatically enclosed in ^ $ would suffice.

[rragan]

1 hour ago, barefootguru said:

Yeah, Google (actually DuckDuckGo) confirms that NSPredicate has to match the entire string.  I do think this will trip people up, as no ^ $ is present.

I like @rragan‘s idea of stripping ^ $ if present and enclosing the string in .* but maybe that’s too complicated, and an explanation on the screen saying the expression is automatically enclosed in ^ $ would suffice.

Or say, if you want your regex as written, it must start with ^ and end with $.  Then only add the .* when no ^$ is present. 

[Nic Hubbard]

2 hours ago, rragan said:

Or say, if you want your regex as written, it must start with ^ and end with $.  Then only add the .* when no ^$ is present. 

I could check to see if the regex string starts with .*, if it does but does not end with .* that could be added automatically.

What other examples are you seeing that aren't working as expected?

[rragan]

Any regex that has the first match part way into the string and not end with .* needs a .* at both ends. I just tried ^abc$  matching abc as the whole string. Adding .* to the start and the end and it still matched so this odd case that worried me is fine. 

It looks like if it does not start with .* then add .* to the start. If it does not end with .*, add that to the end. I believe this action will make regex work like users expect it to. 

[barefootguru]

I think you need:

1. if starts with ^ remove it
2. if ends with $ remove it
3. add .* to start
4. add .* to end

I can't think of a situation where that'll throw the RE out, but as @rragan says, it's a complicated beast 

(Was responding to Nic's earlier post which has been deleted)

[Nic Hubbard]

2 hours ago, rragan said:

Any regex that has the first match part way into the string and not end with .* needs a .* at both ends. I just tried ^abc$  matching abc as the whole string. Adding .* to the start and the end and it still matched so this odd case that worried me is fine. 

It looks like if it does not start with .* then add .* to the start. If it does not end with .*, add that to the end. I believe this action will make regex work like users expect it to. 

I have found a new way to use an updated method NSRegularExpression which has much more control. In testing this seems to fix @rragan's original .* issue and I hope others. I am going to release a new beta with this implemented for testing.

After more testing it appears it also uses the ICU implementation, which I expected, I just thought it would fix some of the issues.

Back to implementing the suggestion by @barefootguru and @rragan.

[rragan]

Much better in Beta 12