Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zerver: Add support for spoilers. #15281

Closed
wants to merge 1 commit into from

Conversation

opheliasdaisies
Copy link
Contributor

Co-authored-by: Dylan Nugent [email protected]
@dylnuge

Resolves #5802

Adds support for containing text content inside an initially collapsed spoiler block, using the following syntax:

```spoiler Spoiler Header
Spoiler Content
```

Spoiler header is optional and will default to the word "Spoiler" if nothing is provided. The spoiler content accepts all standard Zulip formatting including containing multiple paragraphs and code blocks.

Testing Plan:

Manually tested and added automated tests to the fenced_code testing suite.

GIFs or Screenshots:
Screenshot of both expanded and collapsed zpoiler blocks
image-1591731022568

Screenshot of updated documentation
image-1591731125721

A gif of the expand/collapse animation. (Note that the gif is slower than this actually happens)
Screen-Recording-2020-06-09-at-3 25 32-PM

@dylnuge dylnuge force-pushed the zpoilers branch 2 times, most recently from 583b0e0 to 20013b6 Compare June 9, 2020 19:39
@timabbott
Copy link
Member

@opheliasdaisies thanks for working on this! Can you fix all the linter errors? tools/lint will show them all (You should just squash the fixups into your commit and force-push).

Check out our GitHub guide and commit guidelines for more details.

@@ -199,6 +199,7 @@ import "../settings_ui.js";
import "../search_pill.js";
import "../search_pill_widget.js";
import "../stream_ui_updates.js";
import "../zpoilers.js";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cute name, but we should probably just call it spoilers.js for better readability :)

Copy link
Collaborator

@dylnuge dylnuge Jun 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😢 Aww, OK. We can definitely change it (and it shouldn't be anywhere user-facing)

.zpoiler-button:hover {
.zpoiler-arrow {
&:before, &:after {
background-color: grey;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be an HSL color.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed!

margin-top: 2px;
text-align: left;
transform: rotate(45deg);
&:before, &:after {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We generally do a blank line before start of new blocks

"swift": 41,
"tex": 40,
"text": 1,
"vb.net": 45,
"vbnet": 45,
"xml": 1
"xml": 1,
"zpoiler": 50
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably don't need the second zpoiler name, given my suggested rename of the code.

if lang in ('quote', 'quoted'):
return QuoteHandler(processor, output, fence, default_language)
elif lang == 'math':
return TexHandler(processor, output, fence)
elif lang.startswith('spoiler') or lang.startswith('zpoiler'):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be startswith, or can we check for equality here?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops, this is a vestige from an earlier approach we had for same-line spoilers; equality should be fine.

@@ -252,6 +264,37 @@ def done(self) -> None:
self.output.append('')
self.processor.pop()


class ZpoilerHandler(BaseHandler):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this is implied by my comments on naming above, but this should be just SpoilerHandler (etc.).

# No content, do nothing
return
else:
header = self.zpoiler_header if self.zpoiler_header != '' else "Spoiler"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Spoiler" should ideally probably be tagged for i18n. Probably the nicest way to do this is for this logic to live client-side, so that we translate "Spoiler" in the client-side UI, rather than just using the language of whoever posted the spoiler message.

header_text = header
end_header_start_content_html = '</div><div class="zpoiler-content' \
' zpoiler-content-closed" aria-hidden="true">'
footer_html = '</div></div>'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should do the HTML generation using the ElementTree API, like the rest of our backend markdown processor does. (bugdown/__init__.py).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this won't work here (and currently is not used for the latex handling above either) because we need the output of that to partially be treated as already processed HTML (and not rendered as plain text by bugdown) and the part of it (the header and content text) to still be processed by bugdown.

This is why we're calling the stash function (here and in the JS version of this code) on the HTML but not on the header or content text. I may have missed a feature available in ElementTree, though!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, makes sense. We don't do so in the adjacent code for similar features, so I think you're right.

"name": "zpoilers_fenced_spoiler",
"input": "```spoiler header\ncontent\n```\noutside spoiler\n",
"expected_output": "<div class=\"zpoiler-block\"><div class=\"zpoiler-header\"><a class=\"zpoiler-button\" aria-expanded=\"false\"><span class=\"zpoiler-arrow zpoiler-button-closed\"></span></a>\n\n<p>header</p>\n</div><div class=\"zpoiler-content zpoiler-content-closed\" aria-hidden=\"true\">\n\n<p>content</p>\n</div></div>\n\n<p>outside spoiler</p>",
"marked_expected_output": "<div class=\"zpoiler-block\"><div class=\"zpoiler-header\"><a class=\"zpoiler-button\" aria-expanded=\"false\"><span class=\"zpoiler-arrow zpoiler-button-closed\"></span></a>\n\nheader\n\n</div><div class=\"zpoiler-content zpoiler-content-closed\" aria-hidden=\"true\">\n\ncontent\n\n</div></div>\n\n<p>outside spoiler</p>"
Copy link
Member

@timabbott timabbott Jun 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a few more tests:

  • A spoiler inside a block quote.
  • A spoiler where no heading was provided.
  • A spoiler with markdown syntax in the header (E.g. an emoji and a link).
  • A spoiler with a link to an image. (I think there's a decent risk we will shown the image preview outside the spoiler, which might be regrettable, because those appear at the end of a message). I have ideas for how to fix that, but they're somewhat annoying; we may just need that to be a known problem. But we should have a test confirming that we generate the known problem :/.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just checked on this. Links to images that end in one of the extensions in the is_image function in bugdown still work fine, because they're expanded into an inline image. This is an issue for other pages which unfurl content (in a format similar to open graph cards) and append it to the bottom. Zulip appears to have specific logic to unroll Twitter and Youtube links; as far as I can tell, those are the only two sites that it does this for, but those links are definitely affected and appear outside the spoiler block.

We'd be fine just testing the known problem for the time being. What are your ideas for fixing this in the future?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My best idea for fixing it is to have the logic that adds the previews look at whether the link was within a spoiler_block via the tree structure, and if so, append to end of that rather than the end of the message.


// Modify ARIA roles for screen readers
$(this).attr("aria-expanded", "true")
zpoiler_content.attr("aria-hidden", "false")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really appreciate that you did the ARIA roles for this! :)

var zpoiler_height = zpoiler.prop('scrollHeight');
zpoiler.height(zpoiler_height + "px");
zpoiler.removeClass("zpoiler-content-closed");
zpoiler.addClass("zpoiler-content-open");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a brief comment belongs here that this is what triggers the animation for it to open?

const toggle_button_html = '<a class="zpoiler-button" aria-expanded="false"><span class="zpoiler-arrow zpoiler-button-closed"></span></a>'
const header_text = header
const end_header_start_content_html = '</div><div class="zpoiler-content zpoiler-content-closed" aria-hidden="true">'
const footer_html = '</div></div>'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible that this would read better as a handlebars template; feel free to convert it to that.

A few notes on the HTML here, which will be an important and hard-to-change part of the Zulip API, since messages will be stored in the database indefinitely once rendered with a given HTML schema.

  • I'd like to avoid having UI details like the zpoiler-content-closed classes present in the base HTML; we should omit them here. It's possible that adjustments to the CSS to assume a spoiler_content is closed unless specified otherwise is the cleanest solution here (I think it would save code to do so, and just have a class for open spoilers); if not, we can add them before inserting into the DOM in static/js/rendered_markdown.js.
  • I want it to be easy to change the precise HTML for the toggle UI as well, which we can definitely insert via rendered_markdown.js.

I think that means the HTML would be something like this:

<div class="spoiler-block">
    <div class="spoiler-header">{{header}}</div>
     <div class="spoiler-content" aria-hidden="true">{{content}}</div>
</div>

@andersk do you have any further thoughts on how this should be represented as HTML? I could imagine deciding to leave the aria-hidden detail to clients as well.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We handled most of this. I don't think we can do a handlebars template here while also handling the concern around ensuring links/emoji in headers show up properly; the result of the template (markdown and all) would need to be passed to the client side stash function. We've instead cleaned up this function so the header and content don't need to be escaped, which I think should be more readable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, sounds good. With the simpler HTML, this may be fine anyway.

@timabbott
Copy link
Member

timabbott commented Jun 10, 2020

@opheliasdaisies as you saw, I just posted a bunch of comments on the details of this, but overall this looks really promising. We're hoping to release Zulip 2.2 in the next few weeks; but I think it's possible this could be merged for that release, at least as a beta.

The main thing that we might find difficult to address over the next week or two is the issue of inline image previews for links to images/uploaded files. I'd be happy to merge this without resolving that (leaving it for a follow-up issue), since I think the fix for that issue will be mostly orthogonal to the work in this PR.

Let me know when you're ready for another review!

@opheliasdaisies
Copy link
Contributor Author

@timabbott Thanks! We already fixed all the linting errors, and will let you know when we're ready for another pass.

We'll take a look at the inline image previews, but may take you up on leaving that as a follow-up issue.

prepend: function (arg) {
html = arg + html;
return self;
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@showell can you review these test framework changes?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with these additions to zjquery, although I kinda wonder if we actually need append and prepend for the client-side manipulation here? Instead of appending/prepending around the spoiler header, I wonder if it makes more sense for the servers-side markdown to provide the correct structure in the first place? And then we can use .html() to replace the guts? This feels like it would be less brittle.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want to keep the server-provided HTML relatively simple, since that's very hard to change; one can imagine significant styling changes that we can make with the current model that would be harder if we put the HTML for the button UI in the rendered markdown HTML.

@timabbott
Copy link
Member

timabbott commented Jun 12, 2020

These generally look good; from my perspective, it'd be best to squash the fixes into the original commits.

Check out the Zulip commit guidelines for more details: https://zulip.readthedocs.io/en/latest/contributing/version-control.html

@dylnuge
Copy link
Collaborator

dylnuge commented Jun 12, 2020

Squashing is the plan (hence the tosquash label), just didn't want any comments on intermediate diffs or test results to disappear from force pushing while we were in review!

@dylnuge
Copy link
Collaborator

dylnuge commented Jun 12, 2020

Tests are passing and we should be ready for another round of review! Notably I think replacing the HTML in fenced_code with Handlebars/ElementTree won't work right because the inner content needs to be treated as unprocessed markdown. Everything else discussed should be handled now.

// If a spoiler block has no header content, it should have a default header
// We do this client side to allow for i18n by the client
if ($.trim($(this).html()).length === 0) {
$(this).append(i18n.t('Spoiler'));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just use .html() here?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That should work here, sure!

}

// Add the expand/collapse button to spoiler blocks
const toggle_button_html = '<a class="spoiler-button" aria-expanded="false"><span class="spoiler-arrow spoiler-button-closed"></span></a>';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it just make sense to use a template here to build up the html that goes inside of <div class="spoiler-header">whatever server sends</div>? And then we can replace with a single call to $('.spoiler-header').html(html)?

And the server could possibly put some fallback value in the div like _("Spoilers not supported on this client").

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(the reason I make the above suggestion is that I am always a little worried about append/prepend causing tiny regressions)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the message sender specifies header text then the header stored on the server won't be empty and the contents will matter.

We could still replace it without using prepend in some ways; for instance, we could have the server add in a <div class="button-placeholder"> or something similar.

What are your concerns with append/prepend?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be clear, I don't have any show-stopping concerns here. I just feel like sometimes append can be slightly harder to understand and/or easier to break.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the template ends up being messy, and this logic does do precisely what we want. I do wish we could just use .text() to replace the text in a paragraph. Would it make sense to have an empty <p></p> tag be the base header content in our markdown system, rather than having nothing inside? That way, this logic could expect that p tag to be there, and just do $(".spoiler-header p").text(i18n.t("Spoiler")).

(We generally try to avoid using .html() where we can, because it's a place you have to think about malicious HTML.

@zulipbot
Copy link
Member

Heads up @opheliasdaisies, we just merged some commits that conflict with the changes your made in this pull request! You can review this repository's recent commits to see where the conflicts occur. Please rebase your feature branch against the upstream/master branch and resolve your pull request's merge conflicts accordingly.


if (spoiler_content.hasClass("spoiler-content-open")) {
// Content was open, we are collapsing
arrow.addClass("spoiler-button-closed");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spoiler-button-closed class doesn't do anything, so I removed it.

background-color: hsl(0, 0%, 50%);
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I migrated these to use SCSS nesting.

</div>
</div>
</td>
</tr>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the example to be more focused on explanation (no mention of cars).

We should probably do a follow-up project to avoid copying rendered markdown into a static HTML file. In an idea world, we'd just have a bit of JavaScript with a handlebars template to just render the examples using the markdown processor; I think all of the examples here should agree with the backend rendering and let us delete all the generated code here. Opened this as #15375.

@@ -159,7 +160,7 @@ EXEMPT_FILES = {
'static/js/user_status_ui.js',
'static/js/zcommand.js',
'static/js/zform.js',
'static/js/zulip.js',
'static/js/zulip.js'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change looks to be in error; I'll remove.

"name": "spoilers_block_quote",
"input": "~~~quote\n```spoiler header\ncontent\n```\noutside spoiler\n~~~\noutside quote",
"expected_output": "<blockquote>\n<div class=\"spoiler-block\"><div class=\"spoiler-header\">\n\n<p>header</p>\n</div><div class=\"spoiler-content\" aria-hidden=\"true\">\n\n<p>content</p>\n</div></div>\n\n<p>outside spoiler</p>\n</blockquote>\n<p>outside quote</p>"
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a test for trying to use the spoiler feature to insert <script> tags; this sort of test is important. It fails with marked right now for whitespace reasons; you should look at whether we can fix the whitespace mismatch between the two processors (or add marked_expected_output, if not possible).

}

exports.initialize = function () {
$("#home").on("click", ".spoiler-button", function () {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be on body, not home; otherwise, it doesn't work in the markdown help. Also needs stopPropagation / preventDefault.

(I fixed both)

@timabbott
Copy link
Member

timabbott commented Jun 14, 2020

I posted a batch of comments; also squashed your commits and pushed back a fixup commit of my own that has the changes I wanted to make during review. A few notes on further fixes:

  • Frontend tests will fail; see my comment on the script tags change above.
  • Can you update the Help Center docs based on zerver: Add support for spoilers. #15281 (comment) as well? And the screenshots, which are the last relic of the zpoiler name :)
  • The click area for expanding the spoiler is way too small. I think we probably want it to be the entire header, as long as you're not (A) clicking something that has its own UI (E.g. a link in the header) or (B) doing a selection, similar to the "Stream description" area at the top of the page. (B) can be checked with if (document.getSelection().type === "Range"); we do this in like 3 other places, which are likely good references for this change.
  • Is it intentional that the animation to close the spoiler starts with jumping it a bit bigger? Animations aren't my thing so maybe that's a popular pattern, but it was definitely something I noticed.
  • I opened Add support for spoilers zulip-terminal#688 and Add support for new spoiler syntax  zulip-mobile#4155 for the other official clients to support this. The mobile one is quite important; if you're interested in doing the full end-to-end work of this feature, you folks could dig into that.

Otherwise, I'm happy to merge this.

Thanks for all your work on this so far @opheliasdaisies and @dylnuge!

@opheliasdaisies
Copy link
Contributor Author

@timabbott Re: your comment about the click area being too small, we initially had the clickable area be the entire header, but decided to limit it after using it in practice. We thought it made it too easy to accidentally expand the spoiler content. The risk for this would be minimal when using spoilers to discuss movies or TV shows, but we realized that there's a good chance people would also use this feature for content warnings, in which case it seems important to reduce the chance of accidental clicks. We can change it to being the whole header, but I wanted to bring this up first. What are your thoughts?

@timabbott
Copy link
Member

On the expansion UI, I would look at what other tools like StackOverflow that have had spoilers for years do. My experience with them is that there's a large click area for expansion, but I could be misremembering. No need to reinvent the wheel here :)

@timabbott
Copy link
Member

@opheliasdaisies just wanted to check in on this; I'd really like to merge this PR (even if the expansion UI isn't perfect) so that we can get testing in production before the Zulip 2.2 release.

I think we can improve the spoiler-open UI in a follow-up commit; it's unlikely to involve HTML changes, so can easily be done later.

Can you prioritize fixing CI?

@dylnuge
Copy link
Collaborator

dylnuge commented Jun 16, 2020

@timabbott Can you explicitly list out the changes we need to make to merge this? We're going through the stuff you pushed, as I read it, the critical remaining changes are:

  • Fix the XSS test you added to match the frontend renderer's spacing properly
  • Update help center docs to not contain jokes 😆

Is there anything else blocking?

@timabbott
Copy link
Member

timabbott commented Jun 16, 2020

Yes, those are the two blocking issues for merging this. Sorry to kill your jokes! I was amused by them, but every joke in the codebase ends up being a maintenance cost down the line :)

@dylnuge
Copy link
Collaborator

dylnuge commented Jun 16, 2020

No worries; given the trickyness of finding things like zjquery and bugdown I totally understand 😄

@timabbott
Copy link
Member

Yeah, if you're looking for a project after this, renaming zerver.lib.bugdown to zerver.lib.markdown would be a great contribution :). I've been meaning to do that for years.

@dylnuge
Copy link
Collaborator

dylnuge commented Jun 16, 2020

Awesome! We think this should handle everything (assuming tests pass)!

Is there a CLA we need to sign? We couldn't find one in the CONTRIBUTING.md file though we did see some references to a Dropbox one potentially being replaced?

@timabbott
Copy link
Member

We don't require signing a CLA to contribute :).

@timabbott
Copy link
Member

Merged as 1cb0406, after adding a bit more detail to the commit message (not much felt required, since the documentation itself explains the feature)

Thanks @opheliasdaisies!

@timabbott
Copy link
Member

Opened #15414 to track the spoiler block open/close UI issue to make sure we don't lose track of that.

And I think you haven't replied to this question yet:

Is it intentional that the animation to close the spoiler starts with jumping it a bit bigger? Animations aren't my thing so maybe that's a popular pattern, but it was definitely something I noticed.

Depending on the answer, we may want to open another issue there as well.

Huge thanks for building this @opheliasdaisies and @dylnuge!

@timabbott
Copy link
Member

And opened a blocker issue that I realized when thinking through other things one surely wants in terms of this feature's behavior: #15416

@opheliasdaisies @dylnuge can you folks work on that? Feel free to open a PR as soon as you have at least one of the three notification types updated.

@opheliasdaisies
Copy link
Contributor Author

@timabbott We're really excited to have this merged in! The animation to close the spoiler block isn't as we intended. It definitely used to not do that, and something we changed along the way must have altered its behavior. We're up for fixing that.

@timabbott
Copy link
Member

timabbott commented Jun 17, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Hide text (spoiler) on message
5 participants