Web and Browsers

Apart from the terminal, the web browser is a tool you will find yourself spending significant amounts of time into. Thus it is worth learning how to use it efficiently and

Shortcuts

Clicking around in your browser is often not the fastest option, getting familiar with common shortcuts can really pay off in the long run.

Search operators

Web search engines like Google or DuckDuckGo provide search operators to enable more elaborate web searches:

More through lists are available for popular engines like Google and DuckDuckGo

The searchbar is a powerful tool too. Most browsers can infer search engines from websites and will store them. By editing the keyword argument

For example you can make so that y SOME SEARCH TERMS to directly search in youtube.

Moreover, if you own a domain you can setup subdomain forwards using your registrar. For instance I have mapped https://ht.josejg.com to this course website. That way I can just type ht. and the searchbar will autocomplete. Another good feature of this setup is that unlike bookmarks they will work in every browser.

Privacy extensions

Nowadays surfing the web can get quite annoying due to ads and invasive due to trackers. Moreover a good adblocker not only blocks most ad content but it will also block sketchy and malicious websites since they will be included in the common blacklists. They will also reduce page load times sometimes by reducing the amount of requests performed. A couple of recommendations are:

You can find about more addons of this kind here

Style customization

Web browsers are just another piece of software running in your machine and thus you usually have the last say about what they should display or how they should behave. An example of this are custom styles. Browsers determine how to render the style of a webpage using Cascading Style Sheets often abbreviated as CSS.

You can access the source code of a website by inspecting it and changing its contents and styles temporarily (this is also a reason why you should never trust webpage screenshots).

If you want to permanently tell your browser to override the style settings for a webpage you will need to use an extension. Our recommendation is Stylus (Firefox, Chrome).

For example, we can write the following style for the class website


body {
    background-color: #2d2d2d;
    color: #eee;
    font-family: Fira Code;
    font-size: 16pt;
}

a:link {
    text-decoration: none;
    color: #0a0;
}

Moreover, Stylus can find styles written by other users and published in userstyles.org. Most common websites have one or several dark theme stylesheets for instance. FYI, you should not use Stylish since it was shown to leak user data, more here

Functionality Customization

In the same way that you can modify the style, you can also modify the behaviour of a website by writing custom javascript and them sourcing it using a web browser extension such as Tampermonkey

For example the following script enables vim-like navigation using the J and K keys.

// ==UserScript==
// @name         VIM HT
// @namespace    http://tampermonkey.net/
// @version      0.1
// @description  Vim JK for our website
// @author       You
// @match        https://hacker-tools.github.io/*
// @grant        none
// ==/UserScript==


(function() {
    'use strict';

    window.onkeyup = function(e) {
        var key = e.keyCode ? e.keyCode : e.which;

        if (key == 74) { // J is key 74
            window.scrollBy(0,500);;
        }else if (key == 75) { // K is key 75
            window.scrollBy(0,-500);;
        }
    }
})();

There are also script repositories such as OpenUserJS and Greasy Fork. However, be warned, installing user scripts from others can be very dangerous since they can pretty much do anything such as steal your credit card numbers. Never install a script unless you read the whole thing yourself, understand what it does, and are absolutely sure that you know it isn’t doing anything suspicious. Never install a script that contains minified or obfuscated code that you can’t read!

Web APIs

It has become more and more common for webservices to offer an application interface aka web API so you can interact with the services making web requests. A more in depth introduction to the topic can be found here. There are many public APIs. Web APIs can be useful for very many reasons:

function c() {
    url='https://www.google.com/complete/search?client=hp&hl=en&xhr=t'
    # NB: user-agent must be specified to get back UTF-8 data!
    curl -H 'user-agent: Mozilla/5.0' -sSG --data-urlencode "q=$*" "$url" |
        jq -r ".[1][][0]" |
        sed 's,</\?b>,,g'
}

Web Automation

Sometimes web APIs are not enough. If only reading is needed you can use a html parser like pup or use a library, for example python has BeautifulSoup. However if interactivity or javascript execution is required those solutions fall short. WebDriver

For example, the following script will save the specified url using the wayback machine simulating the interaction of typing the website.

from selenium.webdriver import Firefox
from selenium.webdriver.common.keys import Keys


def snapshot_wayback(driver, url):

    driver.get("https://web.archive.org/")
    elem = driver.find_element_by_class_name('web-save-url-input')
    elem.clear()
    elem.send_keys(url)
    elem.send_keys(Keys.RETURN)
    driver.close()


driver = Firefox()
url = 'https://hacker-tools.github.io'
snapshot_wayback(driver, url)

Exercises

  1. Edit a keyword search engine that you use often in your web browser
  2. Install the mentioned extensions. Look into how uBlock Origin/Privacy Badger can be disabled for a website. What differences do you see? Try doing it in a website with plenty of ads like YouTube.
  3. Install Stylus and write a custom style for the class website using the CSS provided. Here are some common programming characters = == === >= => ++ /= ~=. What happens to them when changing the font to Fira Code? If you want to know more search for programming font ligatures.
  4. Find a web api to get the weather in your city/area.
  5. Use a WebDriver software like Selenium to automate some repetitive manual task that you perform often with your browser.

이 페이지를 수정.

CC BY-NC-SA에 따라 라이센스를 부여합니다.