Wiki Escaping - Full UTF8 support in Wiki Names and Wiki Page Names - M. Dent

General: This is the home page of a channel. It is similar to someone's profile "wall" in a social network context. Posts created by the channel are displayed according to the observer's viewing permissions.
Create a Post: If you have permission to create posts on the channel page, then you will see the post editor at the top.

Help

M. Dent

dm42@zot.social

Wiki Escaping - Full UTF8 support in Wiki Names and Wiki Page Names

M. Dent

Mon, 08 Oct 2018 14:00:23 -0400

!Hubzilla Development

I've submitted a merge request that implements a new Wiki escaping mechanism that deals with some of the edge cases that give troubles currently (eg. the & character). The new system also allows all UTF8 characters in WIKI names and WIKI page names.

It should be a "drop-in" replacement and should function fine in place of the current system, but it would be good if some testing could take place before it is integrated. SO... I'm asking for the help of anyone who might be able to run it through some testing.

The merge request is on framagit for review: https://framagit.org/hubzilla/core/merge_requests/1321

show all 11 comments

Mario Vavti

Mon, 08 Oct 2018 14:48:09 -0400

Looks good to me. Are you planing to rework parts of it still? Asking because of the wip status...

M. Dent

Mon, 08 Oct 2018 14:57:14 -0400

Not planning on reworking anything, I just didn't want it immediately integrated.

Mario Vavti

Mon, 08 Oct 2018 15:19:56 -0400

I'll merge into dev if it's then. If everything goes well we might still merge into 3.8RC...

M. Dent

Mon, 08 Oct 2018 15:21:31 -0400

Sounds great!

Mario Vavti

Mon, 08 Oct 2018 15:43:22 -0400

I did some quick testing:
Pagenames containing a ' breaks the markdown editor (those were broken before also) - it breaks the javascript somewhere...
Whitespaces will be displayed as (20) in the browser url (those were displayed as %2B before)
& works but also worked before for me...

M. Dent

Mon, 08 Oct 2018 17:47:58 -0400 last edited: Mon, 08 Oct 2018 17:53:22 -0400

I never got & to work with the old system --- you use PGSQL, though, right? I don't know why that would make a difference, honestly, but that's the only thing I can think of.

A question about the ' character - does it also break the Markdown if a stray ' happens to be on the page somewhere?

It is possible to hardcode a separate filter in the Markdown flow (as well as the HTML and BBCode flow for that matter) to filter things before they go into processing of the formatting. Though that really should be handled by the formatting processor itself.

If there is something specific to LINK formatting that the creata_links() function should be 'aware' of, I think the best way to handle it would be to create a "link_filter" hook in the create_links() function itself and use Hook::insert() to add an appropriate hook just before calling create_links(). That way if new/different page formats come out that have special requirements, they can be easily dealt with.

Since it was broken before, the new code doesn't put us in any worse position than we were in (though, clearly it doesn't solve ALL the problems like I hoped it would, either).

Mario Vavti

Tue, 09 Oct 2018 05:02:48 -0400

No, i am actually using mysql on this server. For the ' issue i think it is just because we probably use the unescaped string in the wiki javascript. It also breaks deleting of a wiki if the wiki name contains a ' because we call the delete function like this function('it's my wiki');.

M. Dent

Tue, 09 Oct 2018 12:16:24 -0400

Ah, sounds like we need to use a different variable or create a structure to escape the pagename before it gets passed to the page. At least it's clearly defined.

M. Dent

Tue, 09 Oct 2018 21:09:10 -0400

@Mario Vavti -- Smarty actually has some built in escaping. It might be good to get into the habit of using the $varname|escape:'quotes' (for display) and $varname|escape:'url' (for URL related stuff) as appropriate when using variables in Javascript calls in templates. I pushed a fix for deleting pages with quotation marks to dev.

Mario Vavti

Wed, 10 Oct 2018 01:15:27 -0400

Yeah we use that at some places already... Thanks for fuxing this...

M. Dent

Wed, 10 Oct 2018 09:59:04 -0400

Nicely done. Thanks for fixing the fix! I missed the 'javascript' option.