Tuesday, March 31, 2015

The story of SailfishOS browser

Why on earth would someone start yet another browser project, when there's so many already out there? Well, because all mainstream browsers are more or less scaled down desktop browsers, than something designed with one handed mobile use in mind.


Looking at the image above, the two examples from the left are impossible to comfortably use without altering your hand grip. Also the reason for such design is obvious just by looking at desktop browser interfaces: they hardly need to be operated with one hand alone. I mean, try lifting your desktop/laptop with one hand, and enter a website address.

Even if you can somehow do it, it's not comfortable; and downscaling that interface design to 5" screens does not change that. You'll still need both hands to operate it, which is an unacceptable requirement for a simple tool that should increase your potential, not decrease it.

And that's where the third example on the right comes to play. Obviously, we're not doing everything by ourselves. We chose to build our web browsing experience on top of Mozilla's Gecko rendering engine, developed for the Firefox browser.

Then, let's look at the two major steps we took before ending up with the current SailfishOS browser design. There's some good things we learned along the way.

Version one

In addition to moving the toolbar down, the first design (click to enlarge) used separate pages for both tabs and website address. The aim was to avoid the performance hungry scenario, where some web content is rendered in the background, a virtual keyboard is open, and there's search suggestions on top of everything.

The toolbar had buttons for back, address field, tabs, reload/stop and forward. In the end, the design was discarded as too complicated to understand. The root cause was in the magnifying glass icon. People didn't associate it to searching the web.

Version two

The next image shows the design that actually shipped with the Jolla phone sales release (click to enlarge). It combined both tab and addres entry into a single page. And instead of a separate address eentering icon, it promoted easier bookmarking.

We got a lot of negative feedback for that design, as it added an extra step to searching something or entering page address, and also made tabs functionality unnecessarily complicated to understand, use and develop.

Version three

The current design brought back separated tabs and address entry points. Tabs still use a page, but the latter one behaves like a toolbar extension, so that you can still see a bit of the webpage underneath. Much more useful functionality was also added throught the expanding/collapsing toolbar.

The feedback has been really good, and although I'm not completely happy about two different gestures to get back to browsing (flick address overlay down to close it vs. flick tabs page right to close it), we're much closer to a modern mobile browser interface.

Something that supports the way your hand works, instead of how you moused around in desktop interfaces few decades ago.

Thanks for reading and see you in the next post. In the meantime, agree or disagree, debate or shout. Bring it on and spread the word.

18 comments:

  1. The flick down extends to the search also. Also the flick down gives you no indication of what you should do to get back to browsing. Until i discover it i thought i was stuck and close to open a TJC bug. The tab page has the indicator so you know you can flick right. I believe the behavior should change to flick right to get back to browsing from every page.

    Apply pressure to appropriate people to get this fixed. :P

    Also i think i asked you again. Could you write a post -if you find some time of course- on what contributions from the community you liked?? Ie recently there was the flat patch which makes the theme flat. Also there are icon sets on TJC and other stuff.

    Regards.

    ReplyDelete
    Replies
    1. True, gestures have the problems that they have no indication by default (all platforms suffer from it, but that's not an excuse). The page navigation is taught in the tutorial, but the overlay part would require an interaction hint to be obvious. Even if SFOS advocates direct manipulation and reversing an animation is usually the way out of a situation, it's not immediately clear as you also noticed. It can be a small thing missing, but it has a big impact.

      Sure, I'll ask if the issue gets some support :)

      Ah yes, now I remember. I think I should write a post about that kind of stuff. There is actually something I wish for.

      Thanks for commenting and reminding me. Cheers :)

      Delete
  2. Hello Jaako,

    I don't want to be rude, but even the version 3 is far from what the rest of sailfish is doing.
    The lower button bar is of heavy use in a browser when using tabs, but it is not in the comfort area...

    I agree that it is not easy to put buttons in the middle of the page, but something else could be done : remove it !
    That is what has been done in the main interface : removing the lower buttons as they are not easy to use. It is even one of the selling points of Sailfish OS.
    If we could use a swipe gesture from the border to access the tabs (browser can be seen as an OS in the OS, with tabs like apps), then it would be a lot more easier to use it. You could even have the same thing than what sailfish OS looks like : swipe from the border (easy!), you get the open tabs view (like the multitasking view), scroll down, you have the favorites. Add to that somewhere the way to type the webpage address, and also the action to add as favorite (not used a lot so can be harder to reach), and you have something that feels like sailfishOS, not something with buttons like the third revision.

    But, (it would be too easy without a "but" ;) ), would it be a good idea to share the border swipe between the OS and the browser... ? We good say that even in a browser page, 2 swipes go to the homescreen (first to tabs, second to home), but this is not that practical...

    This is one of my main complaints about the browser : tabs are not really easy to use. Most of the links I open come from apps (like twitter) or other pages opened from bookmarks, so the progress made on app address is not enough.

    And to be fair, I am using the third iteration as my main browser. If the second iteration was kept I would have stopped (I started using alternatives before the third was released), so it is going in the good direction !

    Keep up the good work !

    Zeta

    ReplyDelete
    Replies
    1. Hi Zeta,

      No problem at all :) Still, to be fair, we shouldn't compare a browser to an OS directly like that. Let me explain.

      First, Sailfish OS window and page control gestures were developed by over 15+ Qt and QML specialist, whereas the browser is being developed by 1-2 persons when they're not occupied by something else. I might post more ambitious design someday, that would be more in line with what I write about comfortable one hand use. Meanwhile, the toolbar is the best we could do with the resources that we have.

      Second, it's not trivial to make webpage content follow user finger the same way silica pages do on native applications (or images in gallery application). It might happen someday, but personally I don't know if that can be done with the Gecko engine developed by Mozilla. It supports discrete mouse gestures (flick first, action will be performed with a small delay), but it's a world apart from its responsive Silica counterpart. Also many mobile sites support horizontal gestures of their own, and that's yet another can of worms.. not to mention where would all the remaining functionality go that is currently housed by the two-story toolbar, if that got removed.

      Third, as you also pointed out, when apps start to use edge swipes to do application specific actions, people will face the same confusing issue as every other platform: you don't know what an edge gesture does. And since edge gestures are the most invisible type of gestures in general, I would advice against making them customized for apps. Take Android and iOS as an example. There's no standard and it comes with the expense of the usability/efficiency of the system as a whole.

      However..

      If the OS features a possibility to lock current app to the foreground (game, video, browser, mediaplayer, camera..) I guess the swipe could then be handled by that app. When user would perform an edge swipe, some form of lock indication would be given so you know what's going on, but you could jump between tabs or close tabs or whatnot. That kind of mode can be useful, but also evil if the indication and feedback are inadequate/missing.

      Thanks for the constructive feedback, I really appreciate the honesty, even if it gets me into trouble at times...

      Take care :)

      Delete
    2. Hello Jaako,

      Thanks again for taking the time to give such a comprehensive answer !

      I thought of it a bit more, and may have found one underlying problem, that is common between the browser and some other apps:
      the app contents needs both horizontal and vertical scrolling.

      When the content only needs vertical scrolling, then gestures to scroll left/right are used to switch between pages (like in settings), or to accept/cancel things.

      The gallery has the same problem, when zooming in a picture, you can not anymore use the left/right gesture to open tabs or come back to the list. The solution used there is that a single tap will spill the screen to show a usual sailfish page in the top half, where gestures work as excepted.

      However, in web browser, tapping is already used as you need to click links or set the focus in form fields, so it can't be used as in the gallery...

      Another solution would be to remove the need to scroll the content left/right. This is something that for example is not needed in websites designed for mobile use. When left/right scrolling is needed it is because you are reading a desktop oriented web page, so obvioulsy not good for experience on mobile.
      There are solutions for this, like the "reader mode" some browsers (or plugins) allow, where it removes everything eye candy to keep only the text and main images. Then, it would be more lightweight and easier to handle left/right gesture for a tabs page (not edge swipe, but simple scrolling).
      This is probably harder to do that my current knowledge would allow, and it would not work for complex websites, but for simple ones it could be great, even as a separate app (keeping the default browser for full websites). A kind of graphical lynx with a silica UI.
      I will try to take a look at how the other browsers/plugins do it, maybe there is already something reusable... ?

      Zeta

      Delete
    3. Hi Zeta, sorry about my late reply, I tried to have a less-technological holiday :)

      You're right, it would need a "reading mode" or similar to work. It would be great if sites would be designed this in mind, instead of implementing their own horizontal gestures-functionalities. Most of the time they're just novelty, instead of really useful.

      Only map canvas is something that breaks it.. needs some thought still. If you happen to run across something, please let me know.

      Thanks for replying. Looking forward for your ideas :)

      Delete
  3. One thing that I hate about the third version is the bookmarks screen. Most of the screen estate is given over to showing the favicons, who uses them anymore? Most of my bookmarks are just the default icon and a tiny bit of the title. It's so much worse than it was in V.2 with the list.

    Apart from that I love the browser, definitely room for improvement in the UI but it almost never has a problem with a webpage, and usually outperforms Safari on my iPad. Can't wait for the tablet to show up :)

    ReplyDelete
    Replies
    1. Hi Tomas,

      I don't like favicons either, they sometimes break or are of low quality. Grid is also unforgiving to long bookmark names. What's good though is that I get 8 bookmarks visible above the keyboard. If I recall correctly, the intent was to fit more bookmarks into a single view, so user would have to scroll less. Let's say that I'm giving the grid a chance :D

      Thanks for the honest feedback. Hope to see you again, and take care :)

      Delete
    2. I would bet that the most usual use-case for mobile browser is to quickly check out your favorite sites. You don't need or want to use keyboard for that since you have your book marks right there, or you would have, if the dam keyboard wasn't in the way.

      The bookmark grid and the keyboard jumping up automatically are an atrocious failure of usability. I'm sorry but there's no other way to put it. These are bugs and need to be fixed rather than given a chance.

      Delete
    3. At first I too didn't like the bookmark grid. But they allow folders the same way as in laucher.

      After the update we got only the simple favicons. But adding a new bookmark I found out that you will get a snapshot of the current screen as the favicon. And before pressing the favorite star you can zoom and adjust the screen content. This way you get your a nice icon.
      At the end I removed all my old bookmarks and readded them and edited also the shown text.
      Now my new bookmark icons in combination of my short texts are good recognizable.
      You will get your own custom favicons this way.

      But adjusting and sorting of the bookmarks could be easier.

      Delete
    4. Hi,

      I agree. I've noticed most people have a habit of frequently going through their favorite sites. There's two clear use-cases: you want to check those sites, or use a preferred search engine. This is the feedback we got from the field (two taps to search or favorites).

      We had a matching version of the browser UX:
      - pull the toolbar up to show bookmarks (no KB)
      - tap site address, to enter text (open KB)

      However, I was told, that it too, was an atrocius failure, and there's no need to support both cases. Naturally, the ability was removed. I agree that even with taller than average toolbar, the error tolerance is not very high, but it wasn't by any means impossible or uncomfortable to perform. These could be easily user customizable (tap shows bookmarks, pull up to search).

      I will discuss these points with our team lead when we have time for browser UX improvements.

      Thanks for the honest feedback. Take care :)

      Delete
    5. For some reason, Anonymous #2 reply was tagged as spam, causing me to miss it completely.

      The idea behind the favorite grid approach is to have an identical usage pattern like in launcher. It still doesn't automatically mean that folders are easy to implement. It just means when they finally are, they feel the same.

      One challenge that remains to be solved, is that favorites have context menu with 5 items, unlike icons in launcher. Maybe it's just another menu item, maybe something else. Let's see.

      That's ingenious trick you've discovered. I like it a lot, thanks for sharing and commenting, take care :)

      Delete
  4. I just want text select and "text zoom-in edit"/ cursor zoom, like OS has (drag you finger on typed text) PLEASE!

    ReplyDelete
    Replies
    1. Hi,

      I would like that as well, but the gecko browser engine we're using is not directly supporting that kind of text selection experience. We're working on it though, and hope that a solution can be found soon.

      Thanks for commenting, take care :)

      Delete
  5. Have you considered dropping tabs from the browser? From my point of view, tabs are something that is needed in a desktop browser but having them in the mobile device is just another instance of "scaled down desktop browser". Sailfish is capable of multitasking, and most websites are fully fledged webapps. It would make sense to have each of the web pages that we have open in the home view, so that you can change between them the same way you change between apps.

    This is similar to what happened with Android apps in Sailfish OS. In the beginning, all running android apps shared the same tile and you had to open that "Android tile" and use a special button to change between them. Now each of them get a tile in the home view. I think the same should happen with webapps: use the mechanism that Sailfish has to switch between applications.

    ReplyDelete
    Replies
    1. Hi Javi,

      I like your way of thinking. The value of tabs can be achieved with either an in-app tabs feature or simply having multiple instances of the browser. The latter would make things simpler for sure. This was exactly how the Nokia N9 browser behaved.

      The reason we opted for single browser window with tabs inside instead, was to allow user have more apps open in the background. The current hard limit of 9 apps is problematic for apps that feature multiple windows.

      Another challenge comes from running low on memory. With tabs inside the browser, we're consuming less memory for having only single app cover window at home (they each have a memory cost). Naturally live tabs are anyway serious memory hogs, and currently the phone can support 2-3 live ones before they have to be reloaded. Making each tab appear as a cover window at home screen, would introduce a serious overhead since modern web pages open links into tabs like crazy.

      Technical problems aside, I think both approaches have their good and bad sides, and even if we're now following everyone else with how tabs are architecturally implemented, we haven't forgot our past.

      Thanks for your thoughtful comment, I hope to see you again in the comments section. Take care :)

      Delete
  6. Don't have the time nor need to add such thorough comments as above, just wanted to mention that I like the v3 Brower UI. Thank you.

    ReplyDelete
    Replies
    1. Brevity is a virtue. Good to hear that we're on the right track. Thanks for taking the time to let me know. Take care! :)

      Delete