How hard can it be to code a feature to let users resize images in a software.

By Arthur Milchior, . Permalink Anki

In this post, I expect to show you why it may be difficult to create a seemingly simple program. In particular, to do it well. I'll show case with the last program I wrote, an add-on for Anki. More precisely, the most wanted add-on for anki, according to the vote of users of anki's subreddit: being able to resize image in the editor. This seems to be a simple add-on; after all, resizing by dragging corner has been done in every editing software for decades[1]. In this post, I intend to document all of the things which made me loose time when I created the add-on "Resize image" for anki. I also created a video showing how the add-on works.

Note

[1] Appart from LaTeX, but let's not consider it.

I'm going to mostly consider the code problem relating to add-ons. This is going to be technical, but I'm going to try to give intuition to people who don't code. I'm going to consider changes in order I made them.

The context

First problem is that I never really used jquery. I did some code modification to create the add-on Uses tex as image in editor/reviewer, to debug HSSM's "Multi-column note editor", and to merge both of them with Frozen fields. But I never really took the time to really understand jQuery. This can be seen because the three above mentionned above writes directly plain html instead of creating jQuery objects. Learning jQuery is not something that I expect to be useful often for me, since there are more modern and better javascript libraries.

How Anki's editor work

To explain the problems I needed to solve, I need to explain how Anki is done. The "editor" is the part of the window (widget)which allow you to create and edit notes. This widget appears in three windows, the one to create notes (called "Add"), the bottom part of window to browse note ("Browse") and the window to edit current note ("Edit Current").

While Anki is done in Qt, for some reason, the editor is done in html+css+js+jQuery[1]. All of those are quite useful to let user easily create card templates. But I don't know why Qt was not sufficient for the graphical interface. In particular each "field" is not actually an input text field as it is standard in HTML. It's a standard div field which is said to be editable. Each time the edition stops for two seconds and each time you leave the current field, its value is sent to Python which saves in the note object.

I see a few advantages with using all of this. Since it's a standard div, you can have any HTML inside of the field, and not only plain text or formatted text. Furthermore, since cards' question/answer are actually coded in HTML, it means you can use any html you want in your note's field.

It also means that each change you want to make to the editor must be in javascript/jQuery/css and not in Python/Qt. Depending on your prefernece as a dev', this can be a good news or a bad news.

The technical problems

Editing an image's size in HTML

As explained above, the field is simply some HTML. In plain english, the editor is just a page, as any web page, except that is is not on internet. This means that I only have to let user edit the size of the image's html tag, and everything would be solved.

A quick google search shows me that the method resizable(https://jqueryui.com/resizable/), a part of jquery-ui, is what I want. And indeed, I simply had to tell the editor to import jquery-ui, and apply resizable to each image. It did seem to work at first. At least as long as the add-on Frozen Field was not used, since this add-on also added images in the editor (a snowflake, whose color indicates whether the field is "frozen"). So instead, I had to restrict the application of resizable to images in fields.

Saving and loading a resizable image

The problem is that "resizable" does more than adding some width/height to an image. It put the image in a div tag ("div" meaning "division of content", e.g. it's a special part of the web page.), and put other div's near to it. Those divs' are saved in the field.

My first intuition told me that it must be those div's which makes the image resizable. If I save those div's with the image, it'll be resizable everywhere. Of course, it may seems nice to imagine that they'll be resizable in the reviewer, on smartphone, etc... but if they are resizable, but the new size is not saved, that'll only create confusion; so it should probably be avoided. You know what ? I was totally wrong. Those div's were not enought to allow images to be resizable. Worst, if you try to apply "resizable" to an image inside of those div, they won't get resizable at all. Instead, you'll see the resizable cursor, but you can't resize them. If you save the broken result and apply resizable to it, you won't even see the cursor anymore. That means that saving those extra div would make the image not resizable !

I should not that jquery-ui's resizable is not so bad; because if you apply resizable to the same image multiple time, it works correctly. The problem occurs when the image is saved and then loaded. I assume that, when you save the image and load it, you have saved a part of the "resizability" of the image; the part which is html, and not the other part which is in javascript/jQuery. And restauring only a part of the "resizability" does not lead to an imagine partially resizable.

The solution seems simple. Resizable has a method called destruct, which is supposed to put everything back as it was before, except that it keeps the new size. That's great, isn't it ? Except that for some reason it does not work. After you apply destruct, the image is not resizable anymore (that's the least I can expect), but the result still contains some useless html content, and because of it, when I save and load the fields again, I can't make the images resizable anymore. So my solution was just to clean all of the html manually. Look for the added div's, the added classes, and style, and remove all of this. That was not hard, I only had to read the code produced by resizable and tells my add-on to remove the div with a class ui-* (and hope that no user really want such a div in its field), the css position and the image's class (hoping that no user/add-on really wanted the image to have any css class.)

The only problem is that I need to do this cleaning each time the field is sent to python. However, this is sent while the field is still being edited. So I first have to clone the field's jQuery node, and then clean the clone.

Having text and image on the same line.

Another unexpected problem. The image can't be on the same line as any text. Indeed, resizable asks for the image/div is to be "block". If I set the image back to "inline", it stops working. Actually, using "inline-block" works. I honestly don't know why.

Text's size was limited by image size.

Let's say you want to add text after an image. Since the image is in a div; your cursor will be in the div, and so suddenly your text will be in here too. So the text will be restricted by the image's new width, and it'll move the image above. Really not something I want. So I had to ensure that actually, the div added for the image are not editable. Which means that in some case, you need to select the image to erase it instead of just pressing delete. I really don't like it, but I don't see any better solution.

Adding new image to a field

Up to now, I mostly dealt with images already in the collection. A note loaded from the collection and which is being edited. But when we create new note, we can't just load image from the collection, we need to incorporate new images. Which leads to new difficulties. Indeed, when the image is added, it is not automatically resizable. In theory, you simply need to call resizable again, but only on the new image(s). In practice, it is actually hard because you don't have any dom object here, but only plain html text to paste. So instead, you select all images in fields which are not already resizable and make them resizable (by the way, the documentation does not state how to check whether an image is not resizable. I used "$img.resizable("instance") == undefined", but I'm not sure it was the correct way to do it.

Images whose width is 0

For some reason, sometime, the image width was put to 0. So the solution I found was to add a little bit of delay before calling resizing. My intuition is that by calling resizing directly, the dom didn't know the size of the image, so assume it's 0, and the resizing then say "okay, so let the image be resizable, and we start with size 0". Ideally, I'd simply state that the image should be resizable once it's loaded. But since I don't actually have a dom/jquery object for the image, I don't see how to do it. I later replaced the delay with the method "ready"; which would have been straightforward for any jQuery dev', but as I said, I was new to it.

Badly resized image

When I tested this add-on, I realized that one important feature was to put back the image to its initial size. Normally, it can be done easily, by setting width and height to "". But for some reason it didn't work. The image got scropped by its previous side. I finally realized that it was because the div containing the image also had the size of the (resized) image. So I had te realize I needed to remove the size indication of the containing div.

Images on top of button

A beta tester told me that when the editor is scrolled down, images were shown over the buttons of the toplevel bar. I never paid attention to it, but when he said it, I undertsand I had to correct it, if I want a good product. The reason this occured is that the top level buttons are in the same html window; as a fixed top level element. And the editor itself was another div element, with a toplevel margin big enough so that nothing gets hidden under the top level bar. The problem is that, for some reason, resizable images get over the fixed element. I thus tried to remove the margin and ensure that the div element was fixed below the top level bar. It was not beautiful because I have no way to know the exact size of the top level bar. I wanted to add a scroll bar to the editor part only, ignoring the top level buttons. But that's not possible, because it seems that in HTML you can add a scroll bar only on element of fixed size. And the size of the editor is not fixed, it depends on the size of the windows. Finally, the solution I found was to put the z-index of the top level bar to 999. I wanted actually to make this change directly to anki's code. Until I discovered that it was already in anki's code; it was introduced in 2.1.20 (which was still in Beta when I started to write this add-on).

Change by 2.1.20

There was one small change in anki 2.1.20 that most people probably did not notice. The width of the images were limited to 90% of its container. Usually it means 90% of the size of the field. If you use a table, then it's 90% of the size of the cell. In my case, it meant 90% of the size of the div, which is used to resize things. Which was a nightmare because I want that the resizable div and the image have exactly the same size. When I updated to last version of anki, it took time to understand why suddenly my add-on broke, and figure out that it was caused by this change in the style sheet. Once found, the solution was easy: put the max-width of each image to be 100%. If I wanted to respect anki change, I could put the max-width of the div to 90%; but I felt like it was a bad idea. After all, I don't want to limit the resizing powers of the users.

Interaction with other add-ons

Two users told me that they wanted my add-on to interact nicely with other add-ons. It makes a lot of sens to want this. There is no reason for a user to choose between a spell checker and being able to resize image. That was actually complex to do, because each add-on need to change the Javascript/css. The way I knew to do it was to replace the method generating the html. Until I discovered the code of Simgunz, which made something quite simple I didn't thought about: injecting js/css after the editor is loaded. In order to automatize it and apply it in other add-ons, I actually create a small library in order to automate this. Even if this is useless, since in 2.1.21 a new method will be introduced in Anki to automate it.

Limit image height

There was one add-on which could not easily interact with my add-on. This add-on Maximum image height in card editor limit the height of images in the reviewer. This ensures that the image does not takes too much space in the editor without limiting the use of big image. That's really a great idea. Except that it's incompatible with my add-on, whichg ensure that we can resize it.

There is no way both add-ons work together; I add to implement the above-mentionned add-on in mine. That was not hard: the only thing it does is change the style so that the max-height of each image is 200px (or any configured value). Applying this rule when my add-on is used lead to a strange result. You could resize images up to height 200, and no more. However, the div which contains the image has no limit. So you can try to resize the image to 400px for example. The image won't actually get resized, but the editor reminds it was resized. That means that while the image is 200 px of height, it takes 400 px of size (which makes the add-on quite useless !). Furthermore, in order to resize again, the cursor needs to be put where the cursor was released the last time, instead of being put at the border of the image.

I followed an advice from aPaci95. One click to activate resizing, and one to deactivate it and put back the height limit. That's still easy to use (I hope). Now that I know what I want to do, I had to do it. And hope, whole new code. The solution was easy, I should state that the max-height of this particular image is 100% when the image is clicked; and then remove the max-height when it's clicked again (so that it takes the same limit as the remaining of the document). Except that when I did that, the image was still seen as resizable when actually it was not. So I add to ensure that the image was made resizable only when it was clicked instead of being made immediately resizable. And reciprocally, once the image was clicked a second time, I should remove the fact that it was resizable. Which was easier said than done, since, as I wrote above, when you remove the "resizability" it left a lot of unwanted css indications in the image, which I had to remove. But I can't remove absolutely everything, because otherwise I could not make the image resizable again (because the "resizable" library recalled that it left some elements and would not add them back, even if I manually removed them.)

Click and double click

I discovered something strange that is already well known. By default, jQuery does not support both click and double click. It seems to me that having different actions when you click or double click on an object is nothing special or new. But not, if you set both click and double click action, and then double click, then the click action is executed twice. I just googled, because I was not the first one to have this problem. I expectd to use this solution except that it does not works. I thus modified it to make it work. The idea is simply, on first click, to state "in 1 second, execute the single click action" and if there is a second click during the second, to state "cancel the previous order, and do the double click action right now".

Crowdfunding

By the way, this add-on is not yet public. You can become a beta-tester immediatly by pledging 15 euros on kickstarter. Otherwise, I'll share it publicly as soon as the goal of 300€ is reached. I should state that I don't think I'll use this add-on myself. I only did it because au lot of people want it. More than 70 people actually. Often my addons are downloaded thousands of time. Given the amount of work it represents, as explained above, I thought that 300€ was actually not a lot as far as developper salary goes, given the number of people who may use it. Actually, I choose 300€ when I didn't care about interacting with other add-ons, I probably would have considered to be not worth my time if I knew how much work interacting with other add-on would take.

One fun problem is that, even if a lot of people want this add-on, I can't reach a lot of them, because crowdfunding are forbidden on anki medical school subreddit.

If the crowdfunding did work, I was hoping to be able to crowdfund more ambitious project, such as a website to collaborate on decks; but I fear that I must realize that it's not possible to sell service to Anki's user, even if a lot of them want them.

Note

[1] I know that js+jQuery may seems redundant. But they are actually a part of the editor which does not use jQuery

Add a comment

HTML code is displayed as text and web addresses are automatically converted.

Add ping

Trackback URL : http://www.milchior.fr/blog_en/index.php/trackback/757

Page top