A lot more is possible than what I present here, I'll try to update this document when I meet something very specific.

Docker is a set of platform as service (PaaS) products that uses OS-level virtualization to deliver software in package called containers. -- Wikipedia

Difference between container and image

A docker image can be compared to a SNAPSHOT which we may meet in VMWare or other VME (Virtual Machine Environment).

On the other hand, a docker container is a running instance of a docker image.

Pull a docker image

By default, docker will work with the central registry which is located at https://index.docker.io.

So, to fetch the latest Archlinux image (for example):

$ docker pull archlinux/base

Pull a docker from another registry

If I had another registry, I could do something like this

$ docker pull docker_mirror.example.org/archlinux/base

Example 1: Image and container usage/interaction

After pulling the Archlinux image locally, it's now possible to run it inside a container.

  • To run and get a pseudo-TTY. In other words, run inside a container and create an interactive bash. Here an example of a session.

    $ docker run --name test -it archlinux/base
    [root@1c6f6c9972f9 /]# pacman -S git
    warning: database file for 'core' does not exist (use '-Sy' to download)
    warning: database file for 'extra' does not exist (use '-Sy' to download)
    warning: database file for 'community' does not exist (use '-Sy' to download)
    error: target not found: git
    [root@1c6f6c9972f9 /]# pacman -Sy
    :: Synchronizing package databases...
    core                                                                                                              133.8 KiB   168 KiB/s 00:01 [#######################################################################################] 100%
    extra                                                                                                            1642.3 KiB   378 KiB/s 00:04 [#######################################################################################] 100%
    community                                                                                                           4.8 MiB   372 KiB/s 00:13 [#######################################################################################] 100%
    [root@1c6f6c9972f9 /]# exit

    Note: As we exited the interactive bash, the container stopped running.

  • To see the list of the running containers.

    $ docker ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                       PORTS               NAMES
  • To see the list of all containers.

    $ docker ps -a
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                       PORTS               NAMES
    1c6f6c9972f9        archlinux/base      "/usr/bin/bash"     9 minutes ago       Exited (130) 2 minutes ago                       test
  • To delete the container.

    WARNING: Be sure that the container is not running.

    $ docker rm test # It's working because we gave the name `test` previously.
    $ docker rm 1c6f6c9972f9 # This is the one to use if no name was set.

Example 2: Image and container interaction while developing

Let's say we have a web application that I want to develop. I will normally use my local Nginx environment. But for learning purpose, let's say I want to develop and have a docker container which will serve my application locally.

My application has one file: index.html

<!DOCTYPE html>
<html>
  <head>
    <title>Lorem Ipsum</title>
    <meta name="viewport" content="width=device-width, initial-scale=1" />
  </head>
  <body>
    <h1>Lorem Ipsum</h1>
    <p>
      Quis nostra cubilia cubilia dapibus consectetur suscipit. Euismod rutrum
      ac odio inceptos vehicula. Conubia accumsan congue fringilla dui lacinia
      est curabitur vel ligula integer est! Dictum nascetur nibh suspendisse
      rutrum bibendum sem senectus quisque. Mus lacus curabitur feugiat taciti
      taciti ac, posuere venenatis porta. Erat aenean himenaeos nisi, placerat
      pellentesque primis. Dictumst class natoque vestibulum consectetur
      posuere, elit consectetur. Libero consequat proin nisl vel viverra pretium
      urna; fames facilisi curabitur velit ligula. Duis etiam facilisis nostra
      orci faucibus consectetur! Placerat morbi vehicula, praesent lectus
      praesent nisi. Eros magna inceptos hendrerit ante. Ultrices purus nibh
      litora.
    </p>
    <p>
      Pulvinar facilisis donec volutpat neque turpis dignissim ad eleifend.
      Nibh; facilisis eget dapibus! Nisl quam sollicitudin himenaeos semper
      euismod eu nisi potenti habitasse faucibus! Conubia est tellus odio dis
      turpis congue tincidunt tellus phasellus, ultrices ornare eros. Nec in
      commodo; tincidunt dis in ac mauris dapibus mus ac habitasse! Fames
      elementum sapien congue duis in lacinia velit nostra viverra elementum
      aenean. Integer iaculis accumsan dictum amet himenaeos odio tortor
      parturient feugiat habitasse mollis eget. Torquent ornare ligula sagittis
      felis augue euismod mauris. Scelerisque dictum facilisi lectus taciti.
      Mauris platea.
    </p>
    <p>
      Elit duis odio pulvinar egestas nulla accumsan tincidunt ut suspendisse
      cras orci! Platea aliquam morbi tristique nec, lacinia sagittis proin
      porta magna cras nec praesent! Sit elementum at nulla quis gravida. Orci,
      ridiculus suscipit pretium! Sapien cursus bibendum congue tincidunt est
      posuere class nascetur. Sed rutrum egestas curae; sapien facilisi vehicula
      tempor non et vitae. Senectus montes ad scelerisque sociis non. Fermentum
      venenatis nisi interdum augue dui. Vehicula facilisi phasellus hac sit
      sociosqu ullamcorper justo aliquet dapibus nec. Laoreet arcu dictum
      egestas, primis sapien. Integer commodo senectus, luctus tempor nullam
      natoque suscipit.
    </p>
  </body>
</html>
  1. I pulled the latest Nginx image.

    $ docker pull nginx:alpine
  2. I ran a new container with everything I need.

    $ docker run --name lorem_ipsum -v ${PWD}:/usr/share/nginx/html:ro -p 8888:80  -d nginx
    20f5b2183a22bd1cadbaf9953ba9376f7c2c5d2c9ec61f9e1cb17483cd377075

    Notes:

    • The -v stands for --volume. What I'm saying to docker is that I want to current directory to be mounted as /user/share/nginx/html in the container's filesystem.
    • The :ro stands for read-only. That means that I forbid any modification to the mounted volume.
    • The -p stands for --port. What I'm saying to docker is that when a request comes through the 8888 port, it redirects to the 80 port inside the docker container.
  3. I checked that it's running.

    $ docker ps
    CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                  NAMES
    20f5b2183a22        nginx               "nginx -g 'daemon of…"   3 minutes ago       Up 4 minutes        0.0.0.0:8888->80/tcp   lorem_ipsum
    
  4. I checked that I could access with curl through http://host-ip:8888.

    $ curl http://host-ip:8888
    <!DOCTYPE html>
    <html>
       <head>
           <title>Lorem Ipsum</title>
           <meta name="viewport" content="width=device-width, initial-scale=1">
       </head>
       <body>
           <h1>Lorem Ipsum</h1>
           <p>
           Quis nostra cubilia cubilia dapibus consectetur suscipit. Euismod rutrum ac odio inceptos vehicula. Conubia accumsan congue fringilla dui lacinia est curabitur vel ligula integer est! Dictum nascetur nibh suspendisse rutrum bibendum sem senectus quisque. Mus lacus curabitur feugiat taciti taciti ac, posuere venenatis porta. Erat aenean himenaeos nisi, placerat pellentesque primis. Dictumst class natoque vestibulum consectetur posuere, elit consectetur. Libero consequat proin nisl vel viverra pretium urna; fames facilisi curabitur velit ligula. Duis etiam facilisis nostra orci faucibus consectetur! Placerat morbi vehicula, praesent lectus praesent nisi. Eros magna inceptos hendrerit ante. Ultrices purus nibh litora.
           </p>
           <p>
           Pulvinar facilisis donec volutpat neque turpis dignissim ad eleifend. Nibh; facilisis eget dapibus! Nisl quam sollicitudin himenaeos semper euismod eu nisi potenti habitasse faucibus! Conubia est tellus odio dis turpis congue tincidunt tellus phasellus, ultrices ornare eros. Nec in commodo; tincidunt dis in ac mauris dapibus mus ac habitasse! Fames elementum sapien congue duis in lacinia velit nostra viverra elementum aenean. Integer iaculis accumsan dictum amet himenaeos odio tortor parturient feugiat habitasse mollis eget. Torquent ornare ligula sagittis felis augue euismod mauris. Scelerisque dictum facilisi lectus taciti. Mauris platea.
           </p>
           <p>
           Elit duis odio pulvinar egestas nulla accumsan tincidunt ut suspendisse cras orci! Platea aliquam morbi tristique nec, lacinia sagittis proin porta magna cras nec praesent! Sit elementum at nulla quis gravida. Orci, ridiculus suscipit pretium! Sapien cursus bibendum congue tincidunt est posuere class nascetur. Sed rutrum egestas curae; sapien facilisi vehicula tempor non et vitae. Senectus montes ad scelerisque sociis non. Fermentum venenatis nisi interdum augue dui. Vehicula facilisi phasellus hac sit sociosqu ullamcorper justo aliquet dapibus nec. Laoreet arcu dictum egestas, primis sapien. Integer commodo senectus, luctus tempor nullam natoque suscipit.
           </p>
       </body>
    </html>
  5. I could then continue the development. I removed the last paragram.

  6. The changes were active (directly) without any change to the container.

    $ curl http://host-ip:8888
    <!DOCTYPE html>
    <html>
       <head>
           <title>Lorem Ipsum</title>
           <meta name="viewport" content="width=device-width, initial-scale=1">
       </head>
       <body>
           <h1>Lorem Ipsum</h1>
           <p>
           Quis nostra cubilia cubilia dapibus consectetur suscipit. Euismod rutrum ac odio inceptos vehicula. Conubia accumsan congue fringilla dui lacinia est curabitur vel ligula integer est! Dictum nascetur nibh suspendisse rutrum bibendum sem senectus quisque. Mus lacus curabitur feugiat taciti taciti ac, posuere venenatis porta. Erat aenean himenaeos nisi, placerat pellentesque primis. Dictumst class natoque vestibulum consectetur posuere, elit consectetur. Libero consequat proin nisl vel viverra pretium urna; fames facilisi curabitur velit ligula. Duis etiam facilisis nostra orci faucibus consectetur! Placerat morbi vehicula, praesent lectus praesent nisi. Eros magna inceptos hendrerit ante. Ultrices purus nibh litora.
           </p>
           <p>
           Pulvinar facilisis donec volutpat neque turpis dignissim ad eleifend. Nibh; facilisis eget dapibus! Nisl quam sollicitudin himenaeos semper euismod eu nisi potenti habitasse faucibus! Conubia est tellus odio dis turpis congue tincidunt tellus phasellus, ultrices ornare eros. Nec in commodo; tincidunt dis in ac mauris dapibus mus ac habitasse! Fames elementum sapien congue duis in lacinia velit nostra viverra elementum aenean. Integer iaculis accumsan dictum amet himenaeos odio tortor parturient feugiat habitasse mollis eget. Torquent ornare ligula sagittis felis augue euismod mauris. Scelerisque dictum facilisi lectus taciti. Mauris platea.
           </p>
       </body>
    </html>

Other useful commands

  • Stop a running container.

    $ docker stop test # Replace `test` with the container name or ID.
  • Kill a running container.

    $ docker kill test # Replace `test` with the container name or ID.
  • Delete a container.

    WARNING: The container must be stopped or killed.

    $ docker rm test # Replace `test` with the container name or ID.
  • List all locally available images.

    $ docker images
    $ docker images
    REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE

Cleanup

I could delete everything which was not in use with the following.

$ docker system prune -a
WARNING! This will remove:
  - all stopped containers
  - all networks not used by at least one container
  - all images without at least one container associated to them
  - all build cache

Are you sure you want to continue? [y/N] y
[...]

Links

Here is a list of useful links that provide more information if needed.

Environment creation

  1. I can create a (simple) new environment.
$ conda create --name my_awesome_env_name

  1. I can create a new environment with a specific python version.
$ conda create --name my_awesome_env_name python=3.7.4

  1. I can create a new environment with
    • a specific python version.
    • a list of package to install.
$ conda create --name my_awesome_env_name python=3.7.4 pip jupyter myawesomepackage=1.x

  1. I can create an environment from a YAML file. Which is awesome as it allows us to share environment structure and dependencies across machines.

    Such file look like this:

    name: my_awesome_env_name
    channels:
    # A list of channels to use/look for package from can be given (another awesome feature :=) )
    - defaults
    - conda-forge
    dependencies:
    - python=3.7.4
    - pip
    - flask
    - pip: # All packages which are not available into conda's registry can be listed here.
        - python-dotenv
        - black

    And they can be used like this.

    $ conda env create --file path_to_the_awesome_yaml_file

Environment deletion

For the deletion it is simple.

$ conda env remove --name my_awesome_env_name

Environment update

Sometime, we may need to update the environment.

As example:

  • We don't need an old package
  • A new version of a dependency is available
  • We new a new package

It's as simple as it's creation.

$ conda env update --file path_to_the_awesome_yaml_file --prune

The --prune argument is just the summon of environment management. Indeed it deletes all packages which we don't need :smile:

Environment list

I can get a list of all available environments.

$ conda env list

Package list

I can get a list of the installed package in an environment with the following.

$ conda list --name my_awesome_env_name

Environment clone

Just discovered with the help message that it's possible to clone an environment.

$ conda create --name my_awesome_clone_name --clone my_awesome_env_name

Environment file generation

It's possible to export the currently activated environment into a YAML file. It's practical as we can generate the file and share it.

$ conda env export > path_to_the_new_awesome_yaml_file_to_share

Environment activation

At any time we can switch from an environment to another. But the most important thing to remember: an environment can be activated like this:

$ conda activate my_awesome_env_name

Current environment name and location

To find the currently active environment, we can do this:

$ conda env list | awk '{ if (NF > 0 && substr($1,1,1) != "#" && $2 == "*") print $1 " " $3 }'

Environment deactivation

And to deactivate the currently active environment, it's even easier.

$ conda deactivate

This is something I had been asked for by an Arch newcomer.

The short version

$ pacman -Ql package-name

The long version

$ pacman --query --list package-name

What does the manual say?

-Q, --query

-Q, --query
    Query the package database. This operation allows you to view installed packages and their files, as well as meta-information about individual packages
    (dependencies, conflicts, install date, build date, size). This can be run against the local package database or can be used on individual package files. In the
    first case, if no package names are provided in the command line, all installed packages will be queried. Additionally, various filters can be applied on the
    package list. See Query Options below.

-l, --list

-l, --list
    List all files owned by a given package. Multiple packages can be specified on the command line.

Let me talk a bit about my latest little project I wrote in Python 3.

Problematic

As I'm part of the team behind Ultimate Host Blacklist along with other similar projects, we often encounter domains which are flagged as INVALID by PyFunceble.

So how can we convert each domain of the INVALID.txt generated by PyFunceble to IDNA format so we can reintroduce them for testing? My response? domain2idna.

Understanding Punycode and IDNA

Before continuing to read, I'll invite you to read the following from charset.com which explain Punycode/IDNA:

Punycode is an encoding syntax by which a Unicode (UTF-8) string of characters can be translated into the basic ASCII-characters permitted in network host names. Punycode is used for internationalized domain names, in short IDN or IDNA (Internationalizing Domain Names in Applications).

For example, when you would type café.com in your browser, your browser (which is the IDNA-enabled application) first converts the string to Punycode "xn--caf-dma.com", because the character 'é' is not allowed in regular domain names. Punycode domains won't work in very old browsers (Internet Explorer 6 and earlier).

Find more detailed info in the specification RFC 3492.

With another example, a domain like lifehacĸer.com (note the K) is actually translated to xn--lifehacer-1rb.com. You may not encounter those kinds of domains in your daily navigation over the Internet but when coming to hosts file, we encounter them almost everywhere.

Indeed, today IDNA formatted domain are mostly used for phishing like this hacker news article which explain a bit deeper the danger about IDNA.

About domain2idna

Domain2idna can be found on GitHub and is ready to use!

It can be used in two different ways: As an imported module or As a command-line command.

As an imported module

As Python allow an installed module to be imported here is an example of how to use domain2idna into an existing code or infrastructure.

#!/usr/bin/env python3

"""
This module uses domains2idna to convert a given domain.

Author:
    Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom

Contributors:
    Let's contribute to this example!!

Repository:
    https://github.com/funilrys/domain2idna
"""

from colorama import Style
from colorama import init as initiate

from domain2idna.core import Core

DOMAINS = [
    "bittréẋ.com", "bịllogram.com", "coinbȧse.com", "cryptopiạ.com", "cṙyptopia.com"
]

# We activate the automatical reset of string formatting
initiate(True)

# The following return the result of the whole loop.
print(
    "%sList of converted domains:%s %s"
    % (Style.BRIGHT, Style.RESET_ALL, Core(DOMAINS).to_idna())
)

# The following return the result of only one element.
print(
    "%sString representing a converted domain:%s %s"
    % (Style.BRIGHT, Style.RESET_ALL, Core(DOMAINS[-1]).to_idna())
)

That is a simple example to understand how the domain2idna works.

As you can note, domains2idna can return two type: a list or a str. Indeed, because I'll mostly use domain2idna to convert big lists, I wrote domain2idna so it can handle a given list and return a list with the converted domains. In the other side, as most people will want to get the IDNA format of only a domain, domain2idna also return an str if a string is given as input.

As a command-line

This part is less "interesting" but you may find the following usage which explains greatly how it's working.

usage: domain2idna [-h] [-d DOMAIN] [-f FILE] [-o OUTPUT]

domain2idna - A tool to convert a domain or a file with a list of domain to
the famous IDNA format.

optional arguments:
-h, --help            show this help message and exit
-d DOMAIN, --domain DOMAIN
                    Set the domain to convert.
-f FILE, --file FILE  Set the domain to convert.
-o OUTPUT, --output OUTPUT
                    Set the file where we write the converted domain(s).

Crafted with ♥ by Nissar Chababy (Funilrys)

As the conclusion, it was fun to write that little project and I hope that it'll help the Open-Source community!

That's it for the presentation of the project! A detailed code comment/explanation may come soon on the programming section.

Thanks for reading.

Today I challenged myself to compute a way to find if a number is happy or not.

So what is a happy number?

To quote the wikipedia page:

A happy number is defined by the following process: Starting with any positive integer, replace the number by the sum of the squares of its digits in base-ten, and repeat the process until the number either equals 1 (where it will stay), or it loops endlessly in a cycle that does not include 1. Those numbers for which this process ends in 1 are happy numbers, while those that do not end in 1 are unhappy numbers (or sad numbers).

Now that we know what a happy number is let's code !

Let's get a positive number from the user

For this part I will use:

  • input() to get the input from the user.
  • int() to convert the output of input() to integer.
  • abs() to return the absolute value of the given number.
def given_number():
    """
    This function ask the user for a number and return it so it
    can be usable by other functions.

    Returns: int
        The given number.
    """

    # We use while because I want to infinite ask for a number
    # if the user do not give us a number.
    while True:
        # because the use can input anything, I choosed to
        # handle  the exception if the user do not give us a
        # number.
        # This way we keep asking untill the user give us a
        # number.
        try:
            initial_number = abs(int(input("Give us a number ")))
            break
        except ValueError:
            continue

    return initial_number

Spliting digits

Because to compute the happy numbers checking we have to calculate the sum of the square of its digits in base-ten. So, I choosed to directly split each digit of the sequence before doing the calculations.

For this part I will use:

  • int() to convert each digits to integer.
  • str() to convert the given number to string.
  • map() to iterate through the converted string.
def split_digits(digits):
    """
    This function split each digits of a given number.

    Argument:
        - digits: int
            The number to split.

    Returns: list
        A list with each digit.
    """

    # We convert the given number to string first.
    # Then we the iterate over each characters which are
    # converted as integer.
    # As map return a list, and int replace the current
    # iteration to integer, we get the list of each digits
    # of the number.
    return map(int, str(digits))

Calculation of the sum of the square of each digits

For this part I will use:

  • pow() to calculate the square of each digits.
def calculation(digits):
    """
    This function return the calculation of the sum of the square
    of each digits.

    Returns: int
        The result of the recursively sum of squares of each
        digits.
    """

    # This variable is used to store the calculation results.
    result = 0

    # We use for to iterate through the list of digits given by
    # split_digits().
    for digit in digits:

        # We append the square of the current digits to result
        # this way we can return the results once we finished
        # to iterate through the list of  digits.
        result += pow(digit, 2)

    return result

Is a number happy ?

Because I wanted to be eable to check if a number is happy from another script, I choosed to write a function which will tell us if a number is happy (True) or unhappy (False).

As I also want to see the sequence of the results of calculation() when I will work with this function, I introduced between everything a switch which if True will give us a tuple of (True, [results of calculation()]) if the number is happy and (False, [results of calculation()]) if the number is unhappy.

Finally please note that as I did not want to wait for an endless loop, I choosed to check if the result of calculation() was already in our list of results. If it is the case then we have an unhappy number.

For this part I will use:

def is_happy(number, return_sequence=False):
    """
    This function check if a number is happy or not.

    Argument:
        - number: int
            The number to check.
        - return_sequence: bool
            If True we return the sequence of results.

    Returns: bool or tuple
        - True: number is happy.
        - False: number is unhappy.
        - tuple: if return_sequence == True
            - We return (True|False, past_results)
    """

    # This will save the list of previous or past results.
    past_results = []

    list_of_digits = split_digits(number)

    # I choosed to to an endless loop because we do not know
    # where we are going and which path to choose.
    while True:
        current_result = calculation(list_of_digits)

        if current_result not in past_results:
            if current_result != 1:
                list_of_digits = split_digits(current_result)
                past_results.append(current_result)
            elif return_sequence:
                return (True, past_results)
            return True
        elif return_sequence:
            return (False, past_results)
        return False

What if we want to run the scripts ?

Well, to run it as a script with for example python happy_number.py I added the following to the script.

Please note the usage of if __name__ == '__main__': which avoid the script running when we are exporting for example is_happy() for another script or module.

Is this part I will use:

if __name__ == '__main__':
    NUMBER = given_number()

    if is_happy(NUMBER):
        print('%d is a happy number' % NUMBER)
    else:
        print('%d is an unhappy number' % NUMBER)

Final script