UNIX shell

Index

General
  • Linux shell
  • explainshell
  • 10 More Funny and/or Useless Linux Commands
  • Comparison of command shells (wp)
    • bash
    • python
    • ...
  • Bash scripting FAQ
  • BASH Programming HOW-TO
    • functions
    • Exemples / Examples
      • my_func () {
            ...
        }
      • function my_func {
            ...
        }
      • function my_func () {
            ...
        }
      • # call to function
        my_func
      • a=$(my_func)
  • Entorn / Environment
    • set:
      • export TOTO="abc"
    • unset:
      • unset TOTO
      • declare +x TOTO
    • list:
      • export
    • one-shot variable, only for this command:
      • TOTO="abc" my_command ...
    • stored into files:
      • /etc/bashrc
      • /etc/profile
      • /etc/profile.d/toto.sh
        • export TOTO="toto"
      • ~/.bashrc
  • Prompt
  • Shebang
    • #!/bin/bash
    • exit 1 if some command inside the script does not return 0
      • any place in the script
        • #!/bin/bash -e # all commands executed anywhere in the script that return a value
          # different from 0 will make the whole script stop and return 1
      • in some parts of the script
        • #!/bin/bash

          # begin of critical section:
          set -e
          # all commands executed inside this section that return a value
          # different from 0 will make the whole script stop and return 1
          ...
          # end of critical section:
          set +e

          # non-critical calls: commands or functions executed from this
          # point will let the script continue to next commands, despite of
          # the returned value
          ...
  • Debug:
    • from command line:
      • bash -v toto.sh
      • bash -x toto.sh
    • inside the script:
      • #!/bin/bash -x
      • #!/bin/bash
        ...
        # start debug
        set -x
        ...
        # end debug
        set +x
  • $?
    		 returned value (get the exit code of each command in a pipe)
    $@
    		 all command options: $1 $2... (can be programatically set with set -- param1 param2 ...)
    $# number of parameters
    $0
    		 command
    $1
    		 first parameter
    $2
    		 second parameter
    ...
    ${@:2}
    		 from second parameter until the last one
    $$
    		 own pid
    $BASHPID
    		 own pid (like $$, but updated for subshells)
    $!
    		 pid of the latest comand
  • Get the returned value of the latest command:
  • Procés / Process
  • Seguretat / Security

Tests unitaris / Unit tests
  • Bats
    • TAP-compliant (Test Anything Protocol)
    • Info
    • Source
    • Instal·lació / Installation
      • Welcome to bats-core’s documentation!
      • using git submodules in myproject
        • dir structure:
          • my_project/
            • src/
              • myscript.sh
            • test/
              • bats/
              • test_helper/
                • bats-support/
                • bats-assert/
                • common-setup.bash
                  • #!/usr/bin/env bash

                    _common_setup() {
                        # when bats-assert is installed using nvm and npm: npm install -g bats-assert
                        load ${NVM_BIN}/../lib/node_modules/bats-support/load.bash
                        load ${NVM_BIN}/../lib/node_modules/bats-assert/load.bash
                        # get the containing directory of this file
                        # use $BATS_TEST_FILENAME instead of ${BASH_SOURCE[0]} or $0,
                        # as those will point to the bats executable's location or the preprocessed file respectively
                        PROJECT_ROOT="$( cd "$( dirname "$BATS_TEST_FILENAME" )/.." >/dev/null 2>&1 && pwd )"
                        # make executables in src/ visible to PATH
                        PATH="$PROJECT_ROOT/src:$PATH"
                    }
              • mytest.bats
                • setup() {
                      load 'test_helper/common-setup'
                      _common_setup
                  }

                  @test "My first test" {
                      run myscript.sh
                      assert_output 'my output from script'
                      # multiline
                      #assert_output --partial $'first line\nsecond line'
                      # do not forget double quotes, to print the line breaks
                      echo "${output}" >&3
                      echo "# my text!" >&3
                  }
        • my_project must be a git-controlled dir
        • then download git submodules
          • cd myproject
          • git submodule add https://github.com/bats-core/bats-core.git test/bats
          • git submodule add https://github.com/bats-core/bats-support.git test/test_helper/bats-support
          • git submodule add https://github.com/bats-core/bats-assert.git test/test_helper/bats-assert
        • ...
      • using npm
        • global-npm-bats-install-including-bats-assert.md
        • global
          • npm install -g bats
          • npm install -g bats-assert
          • npm install -g bats-file
          • # npm install -g bats-mock # from jasonkarns
          • ...
        • in project (as one of devDependencies in your package.json)
          • cd myproject
          • mkdir node_modules
          • npm install --save-dev bats
          • npm install --save-dev bats-assert
          • npm install --save-dev bats-file
          • # npm install --save-dev bats-mock # from jasonkarns
          • ...
      • Mageia
        • sudo dnf install bats
      • Alma / CentOS
        • sudo dnf install ...
    • Emacs
    • Exemples / Examples
      • setup block
        • #!/usr/bin/env bats

          setup_file() {
              # run once for all tests

              # get the containing directory of this file
              # use $BATS_TEST_FILENAME instead of ${BASH_SOURCE[0]} or $0,
              # as those will point to the bats executable's location or the preprocessed file respectively
              DIR="$( cd "$( dirname "$BATS_TEST_FILENAME" )" >/dev/null 2>&1 && pwd )"
              # make executables in ../ visible to PATH
              export PATH="$DIR/..:$PATH"
          }

          setup() {
              # run before every test

              # https://bats-core.readthedocs.io/en/stable/
              
              # modules installed with nvm and global npm:
              # npm install -g bats
              # npm install -g bats-support
              # npm install -g bats-assert
              load ${NVM_BIN}/../lib/node_modules/bats-support/load.bash
              load ${NVM_BIN}/../lib/node_modules/bats-assert/load.bash

          }

          teardown_file() {
              # run once after all tests
              :
          }

          teardown() {
              # run after every test
              :
          }
      • Test a script
        • @test "Test myscript.sh" {
              run myscript.sh ${parameter1}
          }
      • Test a script with debug
        • @test "Test myscript.sh" {
              run bash -x myscript.sh ${parameter1}
          }
      • Test a function in a library
        • @test "Test mylib.myfunction" {
              source mylib.sh
              run myfunction
          }
      • Skip test
        • @test "Test to be skipped" {
              skip "not ready yet"
              ...
          }
    • Ús / Usage
      • debug
        • display output from script
          • echo "${output}" >&3
        • debug script
          • run bash -x toto.sh
            echo "${output}" >&3
      • run a single bats file
        • bats mytest.bats
      • run all bats files in a dir
        • bats test
      • print raw TAP format
        • bats -t mytest.bats
      • ...

Procés / Process
  • Get the process id of the latest command:
  • & (ampersand)
    • quan es crea un procés fill amb aquesta opció, s'inicia el fill, però torna de seguida el control al pare
    • a la jerarquia de processos, el fill continua penjant del pare
  • wait
    • espera que tots els processos fills hagin acabat (o els hagin matat)
    • si els fills s'engeguen amb & i no es fa un wait, quan el propi procés s'acabi, els fills passaran a ser orfes (penjaran del procés 1)
  • Senyals / Signals (POSIX.1-1990)
    • code
      		 name
      		 description
      		 generated by
      		 captured by "trap ... EXIT"



      • kill -<code>
      • kill -<name>
      • pkill -<name>
      		 others
      0
      		 EXIT

      • exit (in a script)
      • other signals (except KILL)

      1
      		 HUP
      		 Penjat

      		x
      2
      		 INT
      		 ^C
      		CTRL-C x
      3
      		 QUIT


      		-
      4
      		 ILL
      		 La instrucció no és permesa
      		 x
      		x
      5
      		 TRAP
      		 Trampa de traçat/punt d’aturada
      		 x
      		x
      6
      		 IOT
      		 Avortat
      		 x
      		x
      7
      		 BUS
      		 Error de bus
      		 x
      		x
      8
      		 FPE
      		 Excepció de coma flotant
      		 x
      		x
      9
      		 KILL
      		 Matat
      		 x
      		-
      10
      		 USR1
      		 Senyal 1 definit per l’usuari
      		 x
      		x
      11
      		 SEGV
      		 Violació de segment
      		 x
      		x
      12
      		 USR2
      		 Senyal 2 definit per l’usuari
      		 x
      		x
      13
      		 PIPE
      		 -
      		 x
      		x
      14
      		 ALRM
      		 Temporitzador
      		 x
      		x
      15
      		 TERM
      		 Terminat
      		 x
      		x
      16
      		 STKFLT
      		 Fallada de pila
      		 x
      		-
      17
      		 CHLD


      		-
      18
      		 CONT
      		 Continua un procés aturat amb STOP

      		-
      19
      		 STOP
      		 Aturat ()
      		 x
      		 CTRL-Z
      		 -
      20
      		 TSTP
      		 Aturat x
      		x
      21
      		 TTIN
      		 Aturat x
      		x
      22
      		 TTOU
      		 Aturat x
      		x
      23
      		 URG


      		-
      24
      		 GXCPU
      		 S’ha excedit el temps límit de CPU
      		 x
      		x
      25
      		 GXFSZ
      		 S’ha excedit la mida màxima de fitxer
      		 x
      		x
      26
      		 VTALRM
      		 Ha expirat el temporitzador virtual
      		 x
      		x
      27
      		 PROF
      		 El temps de perfilat ha expirat
      		 x
      		x
      28
      		 WINCH


      		-
      29
      		 IO
      		 L’operació d’E/S és possible
      		 x
      		x
      30
      		 PWR
      		 Fallada d’alimentació
      		 x
      		x
      31
      		 SYS
      		 La crida al sistema no és vàlida
      		 x
      		x
    • trap
    • kill
      • quan es mata un procés, els fills (orfes) passen a penjar del procés 1 (/sbin/init)
    • pkill
  • top
  • htop
  • Reinicia un procés si es mor / Restart a process if it dies (respawn)
  • Communication between processes
  • Exemples de processos jeràrquics:
    • ...

Opcions i arguments / Options and arguments

  • Altres llenguatges / Other languages
  • Check the number of arguments (*)
    • #!/bin/bash

      EXPECTED_ARGS=4
      if (( $# != $EXPECTED_ARGS ))
      then
          cat <<EOF
      Usage: `basename $0` arg1 arg2 arg3 arg4
      EOF
          exit 1
      fi
    • #!/bin/bash

      MIN_ARGS=2
      MAX_ARGS=5
      if (( $# < $MIN_ARGS )) || (( $# > $MAX_ARGS ))
      then    
      cat <<EOF
      Usage: `basename $0` arg1 arg2 [arg3] [arg4] [arg5]
      EOF
                exit 1
      fi
    • #!/bin/bash

      # this script does not allow any argument
      if (( $# != 0 ))
      then
          cat <<EOF
      Usage: `basename $0`
      EOF
          exit 1
      fi
    • #!/bin/bash
      MIN_ARGS=2
      MAX_ARGS=8
      #if [ $# -lt $MIN_ARGS ] || [ $# -gt $MAX_ARGS ]
      if (( $# < $MIN_ARGS )) || (( $# > $MAX_ARGS ))
      then
      cat <<EOF
      Usage: `basename $0` [-a|--option_a] [-b|--option_b option_value_b] [--option_c] [--option_d option_value_d] non_option_parameter_1 non_option_parameter_2
      Options:
        -a, --option_a                   explanation for option_a
        -b, --option_b option_value_b    explanation for option_b
            --option_c                   explanation for option_c
            --option_d option_value_d    explanation for option_d Parameters:
        non_option_parameter_1           explanation for non_option_parameter_1
        non_option_parameter_2           explanation for non_option_parameter_2
      EOF
          exit 1
      fi
    • #!/bin/bash

      # defaults
      verbose=0
      flag_a=0
      option_value_b="value_b"
      flag_c=0
      option_value_d="value_d"

      function print_help_and_exit {
          cat <<EOF
      Usage: `basename $0` [options] non_option_parameter_1 non_option_parameter_2
        - non_option_parameter_1        ...
        - non_option_parameter_1        ...

      Options:
        -h, --help                       print this help
        -v, --verbose                    verbose output (default: ${verbose})
        -a | --option_a                  ... (default: ${flag_a})
        -b | --option_b option_value_b   ... (default: ${option_value_b})
        --option_c                       ... (default: ${flag_c})
        --option_d option_value_d        ... (default: ${option_value_d})

      Examples:
      - `basename $0` -h
      - `basename $0` -a -b 3 --option_c --option_d value_d 4 5
      - `basename $0` -a -b 3 --option_c --option_d value_d -- 4 -5

      EOF
          exit 1
      }

      MIN_ARGS=2
      MAX_ARGS=9
      if (( $# < $MIN_ARGS )) || (( $# > $MAX_ARGS ))
      then
          print_help_and_exit
      fi


      # options
      if ! params=$(getopt -o hvab: --long help,verbose,option_a,option_b:,option_c,option_d: -n $0 -- "$@")
      then
          # invalid option
          print_help_and_exit
      fi
      eval set -- ${params}

      while true
      do
          case "$1" in
              -h | --help ) print_help_and_exit;;
              -v | --verbose ) verbose=1; shift ;;
              -a | --option_a ) flag_a=1; shift ;;
              -b | --option_b ) option_value_b="$2"; shift 2 ;;
              --option_c ) flag_c=1; shift ;;
              --option_d ) option_value_d="$2"; shift 2 ;;
              -- ) shift; break ;;
              * ) break ;;
          esac
      done

      # parameters
      non_option_parameter_1=$1
      non_option_parameter_2=$2

      # summary
      echo "Options:"
      echo "  verbose:        ${verbose}"
      echo "  flag_a:         ${flag_a}"
      echo "  option_value_b: ${option_value_b}"
      echo "  flag_c:         ${flag_c}"
      echo "  option_value_d: ${option_value_d}"
      echo "Parameters:"
      echo "  ${non_option_parameter_1}"
      echo "  ${non_option_parameter_2}"

      # do things
      (( verbose )) && echo "$(date '+%Y-%m-%dT%H:%M:%S.%06NZ' --utc) [$(basename $0)]..."
      # ...

      exit 0
    • shift
  • Opcions / Options
    • options and parameters can also be forcely separated by '--' (see rm, getopt)
    • Linux tip: Bash parameters and parameter expansions
    • Using getopts in bash shell script to get long and short command line options
    • tools
      • getopts
        • built-in
        • only single character options: -p -m m_value -d non_option_parameter_1 non_option_parameter_2
        • while getopts ":pm:d" optname
            do
              case "$optname" in
                "p")
                  echo "Option $optname is specified"
                  ;;
                "m")
                  echo "Option $optname has value $OPTARG"
                  ;;
                "d")
                  echo "Option $optname is specified"
                  ;;
                "?")
                  echo "Unknown option $OPTARG"
                  ;;
                ":")
                  echo "No argument value for option $OPTARG"
                  ;;
                *)
                # Should not occur
                  echo "Unknown error while processing options"
                  ;;
              esac
              echo "OPTIND is now $OPTIND"

              next_arguments=${@:$OPTIND}
            done
      • getopt
        • not built-in (provided by e.g. GNU)
          • OSX
            • Install brew
            • brew install gnu-getopt
            • brew link --force gnu-getopt
        • also accepts long option with double-dash prefix: -a|--option_a -b|--option_b option_value_b --option_c --option_d option_value_d non_option_parameter_1 non_option_parameter_2
        • options
          • -o shortops
            • when no short options are available, you must specify an empty set:
              • -o ''
          • -l, --longoptions longopts
            • : -> required argument
            • :: -> optional argument
        • # options
          eval set -- $(getopt -o ab --long option_a,option_b:,option_c,option_d: -n $0 -- "$@")

          while true
          do
            case "$1" in
              -a | --option_a ) flag_a=1; shift ;;
              -b | --option_b ) option_value_b="$2"; shift 2;;               --option_c ) flag_c=1; shift ;;
              --option_d ) option_value_d="$2"; shift 2 ;;
              -- ) shift; break ;;
              * ) break ;;
            esac
          done

          # parameters
          non_option_parameter_1=$1           non_option_parameter_2=$2
        • multiple values
        • if you need to specify newlines as one of the options (e.g.: --user-data-plain '#!/bin/bash -xe\necho "hello world"')
          • # options
            eval set -- $(getopt -o '' --long image-id:,user-data-plain: -n $0 -- "$@")

            while true
            do
              case "$1" in
                  --image-id ) image_id="$2"; shift 2 ;;
                  --user-data-plain ) user_data_plain=$(echo "$2" | sed 's/\\n/\n/g'); shift 2 ;;
                  --do-not-stop ) do_not_stop=1; shift ;;
                  -- ) shift; break ;;
                  * ) break ;;
              esac
            done

Sintaxi / Syntax
  • 3.5 Shell expansions
    • rangs / ranges
      • caràcter únic / single character
        • ls -l name_[0-9].txt
      • 3.5.1 Brace expansion
        • find
        • strings
          • ls -l {a-,b-}name.txt
            • a-name.txt b-name.txt
          • ls -l {a-,b-,}name.txt
            • a-name.txt b-name.txt name.txt
        • integers
          • ls -l name_{00..30}.txt
          • cp -p DSC{00627..00717}.JPG /tmp
        • integers with delta
          • ls -l name_{00..30..2}.txt
    • 3.5.2 Tilde expansion
      • ~
      • ~/foo
      • ...
    • 3.5.3 Shell parameter expansion
    • 3.5.4 Command substitution
      • $(command)
      • `command`
    • 3.5.5 Arithmetic expansion
    • 3.5.6 Process substitution
      • <(list)
      • >(list)
    • 3.5.7 Word splitting
    • 3.5.8 Filename expansion
      • glob
        • shopt -s extglob
          (extglob is on in terminals, but off in scripts)
      • 3.5.8.1Pattern matching

        		• pattern
          		 description

          		*
          		 any string
          ?
          		 single character
          [...]
          		 any one of the enclosed characters
          must be activated with:
          shopt -s extglob
          (extglob is on in terminals,
          but off in scripts)
          		 ?(pattern-list)
          		 zero or one
          *(pattern-list) zero or more
          +(pattern-list) one or more
          @(pattern-list) one
          !(pattern-list) anything except one
  • Conditional constructs
    • List constructs (or, and chains)
    • if
      • Bash conditional expressions (6.4)
        • -a file
          		 True if file exists.
          -b file True if file exists and is a block special file.
          -c file True if file exists and is a character special file.
          -d file True if file exists and is a directory.
          -e file True if file exists.
          -f file True if file exists and is a regular file.
          -g file True if file exists and its set-group-id bit is set.
          -h file True if file exists and is a symbolic link.
          -k file True if file exists and its "sticky" bit is set.
          -p file True if file exists and is a named pipe (FIFO).
          -r file True if file exists and is readable.
          -s file True if file exists and has a size greater than zero.
          ...
        • Simple logical operators in BASH (so)
        • if my_function parameter
          then
            echo "call to my_function returned 0 (ok)"
          else
            echo "call to my_function did not return 0 (not ok)"
          fi
        • # if call to my_function does not return 0, exit with 1 (compact version)
          my_function || exit 1
        • # if call to my_function does not return 0, exit with 1 (long version)
          if ! my_function
          then
              exit 1
          fi
        • # if conversion is ok, remove the input file
          ffmpeg -y -i input.mp4 toto.mp4 1>/dev/null 2>&1
          result=$?
          echo "result: ${result}"
          if ! (( result ))
          then
              # result is ok
              touch ok.txt
              rm input.mp4
          else
              # result is not ok
              touch not_ok.txt
          fi
        • if [ condition ]
          then
            ...
          else
            ...
          fi
        • if [ condition ]
          then
            # do nothing
            :
          else
            ...
          fi
        • if [ ! condition1 ] || [ condition2 ]
          then
            ...
          else
            ...
          fi
        • if [ ! $var_a -eq 0 ] && [ $var_b -ne 0 ]
          then
            ...
          else
            ...
          fi
        • verbose=0
          # check if variable verbose is set to some value
          if [[ "$verbose" ]]
          then
              echo "variable verbose exists and has value $verbose"
          fi
        • verbose=0
          # check if variable verbose is set to some value
          if [ -v verbose ]
          then
              echo "variable verbose exists and has value $verbose"
          fi
        • verbose=0
          # check if content of variable verbose is equal to 1
          if (( $verbose ))
          then
              echo "variable verbose value is 1"
          fi
        • verbose=0
          # check if content of variable verbose is equal to 1
          if (( verbose ))
          then
              echo "variable verbose value is 1"
          fi
        • verbose=1
          (( $verbose )) && echo ...
        • línia única / single line
          • if [[ "Once upon a time..." =~ ^"Once upon" ]]; then echo "found"; else echo "not found"; fi
          • sudo sh -c 'if [[ "Once upon a time..." =~ ^"Once upon" ]]; then echo "found"; else echo "not found"; fi'
        • versió compacta / compact version
          • Replace if … then … fi with && and || in Bash
            • [ $(date +%w) -eq 6 ] && (echo "do something on Saturdays"; do_some_other_stuff; exit 0;) || echo "do different things on other days"
              • final exit 0 is important in order to not to execute the commands after ||
          • print verbose messages:
            • (( $verbose )) && echo ...
          • based on existence of a variable
          • generate options for a call:
            • #!/bin/bash

              do_opt1=0
              do_opt2=1

              function opt1 {
                  (( $do_opt1 )) && (echo "--opt1";exit 0;) || (echo "--no-opt1")
              }

              function opt2 {
                  (( $do_opt2 )) && (echo "--opt2";exit 0;)
              }

              command="my_command $(opt1) $(opt2) --opt3"
              echo $command
              eval $command

              exit 0
            • #!/bin/bash

              do_opt1=0
              do_opt2=1

              #command="my_command $( (( $do_opt1 )) && (echo "--opt1";exit 0;) || (echo "--no-opt1") ) $( (( $do_opt2 )) && (echo "--opt2";exit 0;) ) --opt3"
              echo $command
              eval $command

              exit 0
        • check a flag (e.g. given using getopt)
          • if [ $flag_a -eq 1 ]
        • check if the variable is not empty:
          • if [ "$var" ]
          • if [ -n "$var" ]
        • check if the variable is empty
          • if [ -z "$var" ]
        • check if file exists:
          • if [ -f filename ]
        • Regular expressions
          • Bash Regular Expressions (Linux Journal)
          • Bash in-process regular expressions
          • use regular expression in if-condition in bash
          • regex='(.*)-'
            if [[ $nom_fitxer =~ $regex ]]
            then
                echo $BASH_REMATCH és el que volia
                echo but just ${BASH_REMATCH[1]}
            fi
          • Nota: com posar directament l'expressió regular sense la variable auxiliar $regex?
          • check if a variable content is a member of a list
            • uploadable_extensions="mp4 m3u8 ts"
              item_extension=${item_basename##*.}
                     
              if [[ $uploadable_extensions =~ (^| )$item_extension($| ) ]]
              ...
        • check if a user exists:
          • # create user celery if it does not exist
            if ! id -u celery > /dev/null 2>&1; then
                useradd celery
                echo "Created user celery"
            else
                echo "User celery already exists"
            fi
        • check if user running the script is root:
        • check if the content of a variable matches an element of a given list
        • cadenes / strings
    • case
      • detection of Linux distribution:
        • #!/bin/bash

          distribucio=$(grep '^ID=' /etc/os-release | cut -f2 -d= | tr 'A-Z' 'a-z' | tr -d '"')
          # alternative: distribucio=$(lsb_release -i | awk -F: '{print $2}' | tr -d '\t')
          case $distribucio in
              "mageia")
                  # urpmi ...
                  echo "Detected distribution: $distribucio"
                  ;;              
              centos)
                  # yum install ...
                  echo "Detected distribution: $distribucio"
                  ;;
              debian | ubuntu)
                  echo "Detected distribution: $distribucio"
                  #apt-get install ...
                  ;;
              *)
                  echo "Unknown distribution"
                  ;;
          esac
    • test
      • test expr
      • [ expr ]
      • AND
        • [ expr1 -a expr2 ]
      • OR
        • [ expr1 -o expr2 ]
  • Looping constructs
    • Bash for loop examples
    • Loops for, while and until
    • for
      • #!/bin/bash
        for freq in 514 522 554 570 578 586 618 658 666 682 794 818 842 850 858;
        do
            /usr/bin/dvbstream -f ${freq}000 -qam 64 -gi 4 -cr 2_3 -bw 8 -tm 8 -n 60 8192 -o > /tmp/${freq}_`date +"%Y%m%d-%H%M"`.ts
        done
      • #!/bin/bash
        for f in *;
        do ... ;
        done
      • #!/bin/bash
        for f in *;
        do
        ...
        done
      • #!/bin/bash
        i=0
        while [ $i -lt 5 ];
        do
          echo $i
          let i=i+1
        done
      • #!/bin/bash
        for i in {1..100};
        do
            echo "-------$i--------------"
            ffmpeg -re -i toto.mp4 -c copy -f flv rtmp://localhost/toto/stream01
        done
      • #!/bin/bash
        END=100
        for i in $(seq 1 $END);
        do
            echo $i
        done
      • #!/bin/bash
        max=10
        for (( i=1; i<=$max; i++ ))
        do
            echo $i
        done
    • while
      • While with read (when for has problems with spaces)
      • Bucle sobre json amb jq / Loop over json from jq
      • Wait with a timeout:
        • #!/bin/bash

          timeout=20
          increment=5
          t=0

          while (( t < timeout ))
          do
            echo $t

            if ...
            then
                break
            fi

            (( t+=increment ))
            sleep $increment
          done
      • wait until a file is found:
        • #!/bin/bash

          # seconds
          timeout=20
          increment=5
          filename=/tmp/toto.txt
          t=0

          while [ $t -lt $timeout ]
          do
            echo $t
            # check if the file exists
            if [ -f $filename ]
            then
                echo "detected file"
                break
            fi
            let t=t+$increment
            sleep $increment
          done
      • compact version
        • t=0; timeout=200; increment=2; while (( t < timeout )); do echo $t; if mount -t nfs4 192.168.1.100:/ /mnt/nfs; then break; fi; (( t+=increment )); sleep $increment; done;
    • until ...
      • Respawn
        • # respawn process only if it was terminated with code different than 0
          until myserver
          do
            echo "Server 'myserver' crashed with exit code $?. Respawning.." >&2
            sleep 1
          done

Variables
  • Variables
    • Shell parameter expansion (3.5.3) (Expansions)



    		• example
      conditional print
      		 ${parameter:-word} if parameter exists
        parameter
      else
        word
      ${parameter:?word}
      		 if parameter exists
        parameter
      else
        error: word
      ${parameter:+word} if parameter exists
        word
      else
        (nothing)
      conditional assignation
      		 ${parameter:=word} if parameter does not exist
        parameter=word
      substring
      ${parameter:offset}
      ${parameter:offset:length}
      		 substring of optional given length
      (negative value, preceded by a space, means from the end)
      		 ${parameter:3:2} ${parameter: -3:2}
      # remove last character:
      ${parameter: : -1}
      # get last character:
      ${parameter...}
      names
      ${!prefix*}
      ${!prefix@}
      		 names of variables with names starting with prefix

      ${!name[@]}
      ${!name[*]}

      length
      ${#parameter}
      		 number of characters
      removal
      		 ${parameter/pattern} remove pattern
      ${parameter#word}
      		 remove the shortest matching pattern (expansion of word), applied from the beginning of parameter
      get from the most-left specified pattern (pattern not included) ${parameter#*abc}
      ${parameter##word} remove the longest matching pattern (expansion of word), applied from the beginning of parameter
      get from the most-right specified pattern (pattern not included) ${parameter##*abc}
      ${parameter%word}
      		 remove the shortest matching pattern (expansion of word), applied from the end of parameter
      get until the most-right specified pattern (pattern not included) ${parameter%abc*}
      ${parameter%%word} remove the longest matching pattern (expansion of word), applied from the end of parameter
      get until the most-left specified pattern (pattern not included) ${parameter%%abc*}
      replacement
      ${parameter/pattern/string}
      		 replacement (only first match)
      ${parameter/pattern} remove pattern (all matches)
      ${parameter//pattern/string} replacement (all matches)
      ${parameter//pattern} remove pattern (all matches)
      ${parameter/#pattern/string} replacement: pattern at the beginning
      ${parameter/%pattern/string} replacement: pattern at the end
      case
      		 ${parameter^pattern} lowercase to uppercase (only first character)
      ${parameter^^pattern} lowercase to uppercase (all characters) ${parameter,,}
      ${parameter,pattern} uppercase to lowercase (only first character) gitk
      ${parameter,,pattern}
      		 uppercase to lowercase (all characters)
  • Variable com si fos un fitxer / Variable as it was a file
  • Conversió de base numèrica / Numerical base conversion
  • Integer arithmetics (observe the absence of spaces)

    		• using let
      		 using ((...))
      increment
      		 let var1=$var1+1 (( var1++ ))
      addition
      		 let c=$a+$b c=$((a+b))
      subtraction
      		 let c=$a-$b c=$((a-b))
      ceil(divide/by) let result=($divide+$by-1)/$by; result=$(((divide+by-1)/by))
      random (from 20 to 30)
      		a=$(( ( RANDOM % 10 )  + 20 ))
    • comparison of integers:
      • equal:
        • if [ $variable -eq 2 ]
        • if (( variable == 2 ))
      • not equal:
        • if [ $variable -ne 2 ]
        • if (( variable != 2 ))
      • greater than:
        • if [ $variable -gt 2 ]
        • if (( variable > 2 ))
      • lesser than:
        • if [ $variable -lt 2 ]
        • if (( variable < 2 ))
  • Non-integer arithmetics:
    • Math commands
    • Bash, non-integers and arithmetic
    • total=`echo $var1/$var2 | bc -l`
    • var1=`bc << EOF
      $var1 / $var2
      EOF
      `
    • var1=$(bc << EOF
      $var1 / $var2
      EOF
      )
    • Comparison of non-integers (Shell scripting):
      • if [ `echo "$a < $b" | bc -l` = 1 ] then ...
    • Comprovació que el valor de variable sigui 1.3:
      • if [ `echo "$variable == 1.3" | bc -l` = 1 ]
        then
            echo "la variable si que val 1.3"
        else
            echo "la variable no val 1.3"
        fi  
    • nombre de decimals definit i printf amb punt decimal en lloc de coma (si teniu un locale amb coma decimal) / defined number of decimals and printf with point instead of comma (if you have a locale with decimal comma):
      • export LC_NUMERIC="C"
        a=1.234
        printf "%.2f" ${a}
        # result will be 1.23
      • export LC_NUMERIC="C"
        a=1
        b=3
        echo "$a/$b" | bc -l | xargs printf "%.2f"
        # result will be: 0.33
    • arrodoniment / round
      • export LC_NUMERIC="C"
        a=1.39
        printf "%.1f" ${a}
        # result will be: 1.4
      • echo "23.51" | awk '{printf("%d\n",$1 + 0.5)}'
    • sostre / ceil

Cadenes / Strings

Ordres / Commands

  • Bash reference manual (GNU)
  • Lists of commands (wp: List of UNIX commands):
    • asyncronous: &
    • sequential: ;
    • AND: &&
    • OR: ||
  • Functions
  • Job control:
    • jobs
    • bg
    • fg
    • ...
  • Arrays
    • 6.7 Arrays
    • type
      		 declaration
      		 creation of elements
      		 remove all elements
      		 remove single element
      		 length
      		 print keys
      		 print all values
      		 print keys and values
      		 print single value
      		 print range of values
      indexed
      • declare -a my_array
      • local -a my_array
      • readonly -a my_array
      • my_array=()
      • my_array[0]=tata
        my_array[1]=tete
      • my_array=(tata tete)
      • my_array+=(titi)
      		unset my_array
      		 unset my_array[1]
      • ${#my_array[1]}
      • ${#my_array[@]}
      • ${#my_array[*]}
      • ${!my_array[@]}
      • ${!my_array[*]}
      • ${my_array[@]}
      • ${my_array[*]}
      • printf "%s\n" "${my_array[@]}"
      • for value in "${my_array[@]}"
        do
            echo $value
        done
      • all values separated by specified char (e.g. a comma):
        • echo $(IFS=, ; echo "${my_array[*]}")

      • ${my_array}
      • ${my_array[0]}
      • ${my_array[1]}
      • first element:
        • ${my_array[0]}
      • last element:
        • ${my_array[-1]}
      • from second element to the end:
        • ${my_array[@]:2}
      • 3 elements, starting from the second:
        • ${my_array[@]:2:3}
      • last 2 elements
        • ${my_array[@]: -2}
      • all elements but the last 2:
        • ...
      associative
      • declare -A my_array
      • local -A my_array
      • readonly -a my_array
      • my_array[primer]=titi
      • my_array[segon]=toto


      		for key in "${!my_array[@]}"
      do
          echo "${key}: ${my_array[$key]}"
      done
      • ${my_array[primer]}
      • ${my_array[segon]}
    • Append
      • my_array=()
        my_array+=('tata')
        my_array+=('tete')
    • Remove
    • Sorting
    • Split string:
    • Join array with dots:
      • echo $(IFS='.';echo "${my_array[*]}";IFS=$'')
    • Intersection
    • Unique
  • CPU/memory activity
    • ps
      • fields:
        • VSZ: virtual_size = physical_memory + swap_memory
        • RSS: physical_memory
      • ps --help
      • ps -edalf
      • show threads (like CTRL-H in htop):
        • ps -edalfT
      • by command (http), and sort by RSS
        • ps -yl -C httpd --sort:rss
      • sum the memory ()
        • ps aux | awk '{sum+=$4}; END {print sum}'
    • pgrep
      • Instal·lació / Installation
        • CentOS
          • sudo yum install procps
        • Mageia
          • ...
      • look for a regular expression
        • pgrep -f "my_process.sh"
      • list full command line
        • pgrep -a ...
      • processes with specified parent:
        • pgrep -P <parent_pid>
      • kill
        • pgrep "my_process.sh" | xargs kill -9
        • pgrep -f "some expression" | xargs kill -9
        • processes=$(pgrep -f ...)
          ps ${processes}
          kill -15 ${processes}
    • pstree
      • show also pid:
        • pstree -p <pid>
      • show ancestors
        • pstree -s ...
      • highlight ancestors
        • pstree -h ...
      • ...
    • top
    • atop
      • One-stop performance analysis using atop
      • logs of usage (historic)
        • /etc/sysconfig/atop
          • # interval (default 10 minutes)
            INTERVAL=600
        • /usr/share/atop/atop.daily
          • # delete logfiles older than four weeks
            # start a child shell that activates another child shell in
            # the background to avoid a zombie
            #
            ( (sleep 3; find $LOGPATH -name 'atop_*' -mtime +28 -exec rm {} \;)& )
        • sudo systemctl enable atop
        • sudo systemctl start atop
        • atopsar
          • CPU total:
            • atopsar -c | grep all
        • atop -r /var/log/atop/atop_yyyymmdd
          • t
          • T
          • b: hh:mm
        • search for peaks
          • of memory (minimum free memory):
            • for f in /var/log/atop/atop_*; do atopsar -r $f -m -R 1 | sort -b -h -k 3 | head; done
          • of cpu (maximum all cpu usage):
            • for f in /var/log/atop/atop_*; do echo $f; atopsar -r $f -c -R 1 | grep all | sort -r -b -h -k 3 | head; done
        • prepend every line with analysis date
        • graphs with gnuplot
          • cpu usage
            • atopsar -r atop_20210914 -c | grep all >cpu.txt
            • cpu.plot
              • set timefmt "%H:%M:%S"
                set xtics rotate
                plot "cpu.txt" using 1:3 with lines
          • free memory
            • single day:
              • atopsar -r atop_20210914 -m | awk 'BEGIN {date_stamp=""; } /analysis date: /{date_stamp=$4;} /^[0-9]/ {print date_stamp, $0;}' >free_memory.txt
            • several days:
              • for f in atop_*; do atopsar -r $f -m | awk 'BEGIN {date_stamp=""; } /analysis date: /{date_stamp=$4;} /^[0-9]/ {print date_stamp, $0;}' >>free_memory.txt; done
            • free_memory.plot
              • set timefmt "%Y/%m/%d %H:%M:%S"
                set xtics rotate
                plot "free_memory.txt" using 1:4 with lines
        • ...
    • htop
    • vmstat 1
    • System Activity (KDE)
      • Definit a Arranjament del sistema -> Dreceres i gestos -> Dreceres de teclat globals -> Interfície d'execució d'ordres -> Mostra l'activitat del sistema
        • CTRL+ESC
  • Activitat de fitxers / File activity
  • Activitat de logs / Logs activity
    • logtop
  • Xarxa / Network
    • Network activity
    • ipcalc
      • get CIDR from a given expression
        • eval $(ipcalc -np 172.31.37.147/20); echo $NETWORK/$PREFIX
      • get local CIDR from ip command (only first occurence)
        • cidr=$(ip -o address | awk '$2 !~ /lo/ && $3 ~ /^inet$/ {print $4; exit;}')
          eval $(ipcalc -np $cidr); echo $NETWORK/$PREFIX
      • get ipv4 first address
        • ipv4_address=$(ip -o -f inet address | awk '$2 !~ /lo/ {gsub(/\/24/,"",$4);print $4; exit;}')
    • nc
    • conversion int/ipv4
      • function ipv4_to_int () {
            # convert 1.2.3.4 to 1*256^3 + 2*256^2 + 3*256^1 + 4*256^0
            local ipv4=$1
            ipv4_array=(${ipv4//./ })
            ipv4_hex_array=()
            ipv4_without_dots=''
            for value in ${ipv4_array[@]}
            do
                hex_value=$(printf '%02x' ${value})
                ipv4_without_dots="${ipv4_without_dots}${hex_value}"
                ipv4_hex_array+=(${hex_value})
            done

            # convert base 16 to base 10
            ipv4_10=$((16#$ipv4_without_dots))

            echo "$ipv4_10"

            return 0
        }

        function int_to_ipv4 () {
            # convert 1*256^3 + 2*256^2 + 3*256^1 + 4*256^0 to 1.2.3.4
            value_10=$1

            # convert base 10 to base 16
            value_16=$(printf '%08x' ${value_10})
            length=${#value_16}

            if (( length > 8))
            then
                echo "ERROR: provided value ($value_10, 0x$value_16) is too large to be converted to ipv4"
                exit 1
            fi

            local offset=2
            local ipv4=""
            while (( offset < length+2 ))
            do
                substring_16=${value_16: -${offset}:2}
                substring_10=$((16#$substring_16))

                if (( ${#ipv4} > 0 ))
                then
                    ipv4="${substring_10}.${ipv4}"
                else
                    ipv4=${substring_10}
                fi

                (( offset+=2 ))
            done

            echo "$ipv4"

            return 0
        }
        address_start=224.0.0.1
        address_start_int=$(ipv4_to_int ${address_start})
        echo "${address_start} ${address_start_int}"
        address_stop_int=$(( address_start_int + 2 ))
        address_stop=$(int_to_ipv4 ${address_stop_int})
        echo "${address_stop} ${address_stop_int}"
  • Utilització de disc / Disk usage
  • Substitució / Replacement
  • Elimina el CR
    • sed -e "s!\\r!!g" toto.txt
  • Errors in English:
    • LC_ALL=C <command>
  • File descriptors: stdin, stdout, stderr, ...
  • Power Up Linux GUI Apps (Linux Magazine)
  • Camins i noms de fitxers / Paths and file names
    • URL parsing
    • Python paths
    • Names:
      • my_path=./dir1/dir2/toto.mp4      # ./dir1/dir2/toto.mp4
      • my_dirname=$(dirname $my_path)    # ./dir1/dir2
      • my_rel_dirname=${my_dirname#*\./} # dir1/dir2
      • my_basename=$(basename $my_path)  # toto.mp4
      • my_name=${my_basename%.*}         # toto
      • my_extension=${my_basename##*.}   # mp4
      • my_rel_path=${my_path#*\./}       # dir1/dir2/toto.mp4
      • cd /tmp; mkdir -p dir1/dir2; touch dir1/dir2/toto.mp4
        my_abs_path=$(realpath $my_path)  # /tmp/dir1/dir2/toto.mp4
        my_abs_path=$(eval realpath $my_path) # (use eval when variable contains quotes)
    • Get the present directory:
      • $PWD
    • Get the directory from a filename:
      • dirname $filename
    • Get the abolute path of a file:
      • realpath $filename
    • Get the name of the file (without directory):
      • basename $filename
    • Remove file extension:
      • file_wo_ext=${file%.*}
    • Remove file extension and add a new one ".txt": (Shell parameter expansion)(FAQ)
      • txtfile=${file%.*}.txt
    • Get the file extension: (Shell parameter expansion)(FAQ)
      • extension=${file##*.}
      • to get an empty extension if the file has none:
        • extension_with_point=$([[ "$file" = *.* ]] && echo ".${file##*.}" || echo '')
    • Recursively create the needed directories:
      • mkdir -p ...
  • fitxers ini / ini files
    • Nginx config
    • augtool (Augeas)
      • No tracta qualsevol fitxer: només gestiona els fitxers inclosos per les lenses, sempre i quan siguin correctes
      • Instal·lació / Installation
        • CentOS
          • sudo yum install augeas
        • Mageia
          • urpmi augeas
      • Lenses (modules)
      • Ús / Usage
        • export AUGEAS_ROOT=/tmp/augeas-sandbox
        • augtool -b
          • print /augeas/root
      • Errors
        • Tracking down errors
        • augtool errors (only from version ...?)
        • augtool -b
          • match /augeas//error
          • ls /augeas/files/etc/nginx/nginx.conf/error
          • ...
    • crudini
      • crudini --get toto.in my_section my_key
      • crudini --set toto.in my_section my_key my_value
      • Problems
        • Whitespace around = #33
          • Workaround
            • put your_key=foo. E.g.:
              • sed -i -e '/^#host-name/ s/^#//' avahi-daemon.conf
            • crudini will preserve no spaces around '=':
              • crudini --set avahi-daemon.conf server host-name my_hostname
  • Esborrar línies en blanc / Removing blank lines (*):
    • sed '/^$/d' input.txt > output.txt
    • grep -v '^$' input.txt > output.txt
  • Gotchas
  • Bash completion
    • urpmi bash-completion
    • files:
      • rules:
        • /etc/bash_completion.d/
      • /etc/sysconfig/bash-completion
      • ~/.bash_completion (copied from /etc/skel/.bash_completion)
  • /etc/inputrc
    • Variable settings: set variable value
    • Key bindings:
      • keyname: function-name or macro
      • "keyseq": function-name or macro
        • "keyseq" tecla




          "\e[5~" Re Pág / PgDown
          "\e[6~" Av Pág / PgUp


          "\e[A"
          		 up arrow
          "\e[B" down arrow
    • ...
  • Història / History
  • Comunicació sèrie / Serial communication
    • Linux Serial Console HOWTO
    • 5 Linux / Unix Commands For Connecting To The Serial Console
      • discover

        		• lsusb
          		 dmesg | egrep --color 'serial|tty'
          SparkFun FTDI Basic Breakout - 5V ID 0403:6001 Future Technology Devices International, Ltd FT232 Serial (UART) IC usb 1-1.2: FTDI USB Serial Device converter now attached to ttyUSB0
          YaMoRC YD9100 ID 0403:6015 Future Technology Devices International, Ltd Bridge(I2C/SPI/UART/FIFO)
          		 FTDI USB Serial Device converter now attached to ttyUSB0
          ...

      • configure
        • print configuration
          • setserial -a /dev/ttyUSB0
          • setserial -G /dev/ttyUSB0
      • communicate
        • command usage
          screen screen /dev/ttyUSB0 115200
          cu cu -l /dev/ttyUSB0 -s 115200
          minicom
          putty
          tip tip -115200 /dev/ttyUSB0
  • aleatori / random
  • alternatives
    • Ús
      • alternatives --list
    • Exemples:
  • at
    • instal·lació / installation
      • Alma
        • sudo dnf install at
    • daemon
      • sudo systemctl enable atd
      • sudo systemctl start atd
    • afegeix una tasca / add a job:
      • echo "ls -l >>/tmp/toto.txt" | at 00:12 2024-03-31
    • timedate in [[CC]YY]MMDDhhmm[.ss] format:
      • echo "ls -l >>/tmp/toto.txt" | at -t 202403310012
    • llista de totes les tasques de l'usuari / list of jobs for present users (for all users if run as root)
      • atq
    • mostra els detalls d'una tasca
      • at -c <job_id>
    • esborra una tasca
      • atrm <job_id>
  • atop
  • awk
    • The GNU Awk User’s Guide
    • awk [options] 'pattern1 [&& pattern1b] {action1a;action1b}[;] pattern2 {action2}[;] pattern_start,pattern_stop {action3} ...'
    • Gawk: Effective AWK Programming ()
    • Awk - A tutorial and introduction (UNIX Grymoire)
    • awk (Eric Pement)
    • Add a column and match two files
    • Reimplementation of cut in AWK
    • Exemples / Examples
      • sdp_split.sh
      • nginx logs
      • AWS ALB logs
      • Exemples amb dvbsnoop / dvbsnoop examples
      • Exemples amb OpenSSL / OpenSSL examples
      • els espais dins de les cometes dobles no delimiten camps:
        • gawk ... FPAT ...
        • exemples
          • AWS S3 logs
          • first="first value" second="second value"
            • 'BEGIN {FPAT="([^=]+=\"[^\"]+\")"}; {print $2}'
          • first,"se cond",third
            • 'BEGIN {FPAT="([^,]+)|(\"[^\"]+\")"}; {print $2}'
          • first "se cond" third
            • 'BEGIN {FPAT="([^ ]+)|(\"[^\"]+\")"}; {print $2}'
          • [fi rst] "se cond" third
            • 'BEGIN {FPAT="([^ ]+)|(\"[^\"]+\")|(\\[[^\\[\\]]+\\])"}; {print $2}'
      • Escriu les línies numerades del primer fitxer (NR==FNR) i després les del segon:
        • awk 'NR==FNR{print "first "FNR": " $0;next}{print "second "FNR": " $0}' file1 file2
      • Obté l'enèsima línia (p.ex. 15a) d'un fitxer / Get the nth (e.g. 15th) line of a file
        • awk 'NR==15' toto.txt
        • sed -n '15p' toto.txt
      • Escriu els valors de la segona columna del fitxer, on les columnes estan separades pel caràcter '=':
        • awk -F= '{print $2}' toto.txt
      • Mostra el segon camp ($2) ("xx") de la línia on el primer camp ($1) contingui (~) l'expressió regular "ID_LENGTH" del fitxer .toto.txt (ID_LENGTH=xx)
        • awk -F= '$1 ~ /ID_LENGTH/ {print $2}' toto.txt
      • ...
        • awk -F ":" '{print $1 | "sort" }' /etc/passwd
      • Fes un grep en un altre fitxer de cadascuna de les cadenes trobades:
        • awk -F, '{print $2}' toto.csv | xargs -n 1 -I % grep % tata.txt
      • De la línia que conté la paraula "aquesta", mostra la tercera columna, però sense cap parèntesis / Replace parenthesis to nothing:
        • awk '/aquesta/{gsub(/[()]/,"",$3);print $3}}' toto.txt
      • De la línia (on els camps estan separats pel caràcter =) que conté la paraula aws_access_key_id (aws_secret_access_key), elimina els espais del segon camp i el mostra:
        • awk -F= '/aws_access_key_id/{gsub(/ /,"",$2);print $2}' /etc/aws/credentials
        • awk -F= '/aws_secret_access_key/{gsub(/ /,"",$2);print $2}' /etc/aws/credentials
      • Elimina les cometes simples / Remove sinlge quotes:
        • awk '/.../{gsub(/['\'']/,"",$2);print $2}'
        • awk '/.../{gsub(/[\x27]/,"",$2);print $2}'
      • Elimina les cometes dobles / Remove double quotes:
        • {gsub(/"/, "", $2)}
      • Elimina la primera i segona columnes / Remove first and second columns:
        • awk '!($1="") !($2="")' toto.txt
        • awk '!($1="") !($2=""){print}' toto.txt
      • Elimina les línies que comencen per "/dev"
        • awk '!/^\/dev/' toto_fstab.txt
      • Extreu el text pla d'un fitxer de subtítols vtt:
        • awk -F'>' '/^<c/ {gsub("</c","",$2);print $2}' toto.vtt >toto.txt
      • Blocs separats per una línia en concret
      • Línies al voltant:
        • Print current,next,previous line using awk,sed - BASH
        • Print next few lines after pattern - awk
        • Print the line immediately before a regex, but not the line containing the regex:
          • awk '/regex/{print x};{x=$0}'
        • Print the line immediately after a regex, but not the line containing the regex (only the first occurrence):
          • awk '/regex/{f=1;next}f{print;exit}' toto.txt
        • Print the line immediately after a regex, but not the line containing the regex (all the occurrences):
          • awk '/regex/{f=1;next}f{print;f=0}' toto.txt
      • Awk: print all other columns but not $1, $2 and $3
      • Print directory of files with name ending with index.m3u8 or mp4 (split acts like basename; $(NF-2) takes the third last column)
        • fragment_list=$(awk '/(index.m3u8|mp4)$/ {n=split($(NF-2),a,"/"); print a[n-1]}' $log_path)
      • Esborra línies duplicades / Remove duplicated lines
        • awk '!a[$0]++' toto.txt
      • Obté la suma de la segona columna:
        • awk '{sum+=$2}; END {print sum}'
      • Find the funcion in fabfile.py that contains a put of file mcd.service:
        • awk -v pat="put.*mcd.service" '/^def/ {function_name = $0;} $0 ~ pat {print function_name $0}' fabfile.py
      • HLS
        • mostra les durades dels segments:
          • awk -F: '/^#EXTINF/ {gsub(/,/,"",$2); print $2}' monovariant.m3u8
        • suma les durades de tots els segments:
          • awk -F: '/^#EXTINF/ {gsub(/,/,"",$2); sum += $2} END {print sum}' monovariant.m3u8
        • mostra la suma parcial i el nom de cada segment:
          • awk -F: '/^#EXTINF/ {gsub(/,/,"",$2); sum += $2; print $2 " " sum} /.ts/ {print $1} END {print sum}' monovariant.m3u8
        • troba el segment que conté el punt temporal especificat:
          • awk -F: -v limit=3600 '/^#EXTINF/ {gsub(/,/,"",$2); sum += $2; if (sum>limit) f=1;next} f {print $0; exit}' monovariant.m3u8
      • Get listening port for process snowmix
        • netstat -pnl --ip | awk '$7 ~ /snowmix$/ {gsub(/*.:/,"",$4);print $4}'
      • get disk and mountpoint with a usage greater than a threshold (adding 0 to $2 is necessary to convert it to a number, to make sure that 17>2):
        • usage_threshold=12
          df --local --output=source,ipcent,target | tail -n +2 | awk -v usage_threshold=${usage_threshold} '{gsub(/%/,"",$2)} $2+0 >= usage_threshold {print $1 " " $3}'
  • base64
  • bc
    • ...
  • case
  • cat / EOF
    • to standard output:
      • cat <<EOF
        ...
        EOF
    • to a variable
      • my_var=$(cat <<EOF
        ...
        EOF
        )
    • to a file
      • cat >file_name <<EOF
        ...
        EOF
  • cd
    • va al directori anterior ($OLDPWD) / go to previous directory:
      • cd -
  • chmod
    • chmod -R +X dir_name (-R: recursive; +X: executable/browseable only for directories)
    • change mod of directories:
      • find /var/lib/mysql -type d | xargs chmod 700
    • change mod of files:
      • find /var/lib/mysql -type f | xargs chmod 660
    • recursive sticky group / setgid (only directories)
      • find /absolute/path/to/dir -type d -exec chmod 2770 {} +
  • comm
    • see also: uniq, join
    • print 3 columns
      • first column: only in first file
      • second column: only in second file
      • third column: in both files
    • specified files must be sorted
    • comm f1_sorted,txt f2_sorted.txt
    • comm <(sort f1_unsorted.txt) <(sort f2_unsorted.txt)
    • only entries in f1 (remove columns 2 and 3):
      • comm -23 <(sort f1_unsorted.txt) <(sort f2_unsorted.txt)
    • only entries in f2 (remove columns 1 and 3):
      • comm -13 <(sort f1_unsorted.txt) <(sort f2_unsorted.txt)
    • only common entries (remove columns 1 and 2):
      • comm -12 <(sort f1_unsorted.txt) <(sort f2_unsorted.txt)
    • from variables instead of files:
      • ...
  • cp
    • fes una còpia de seguretat / backup:
      • cp -b
    • filename expansion
    • còpia amb excepcions / copy with exceptions:
      • rsync -a --exclude='dir_to_exclude' source_path destination_path
  • crontab
    • Cron and crontab usage and example
    • crontab guru
    • camps / fields
      • m h  dom mon dow   command
        • m: minut / minute (0-59)
        • h: hora / hour (0-23)
        • dom: dia del mes / day of month (1-31)
        • mon: mes / month (1-12) (Jan-Dec)
        • dow: dia de la setmana / day of week (0-6) (Sun-Sat)
      • Note: implicit AND, except dom-dow, combined with implicit OR
      • exemples / examples
        • cada divendres a les 20:00 UTC / every Friday at 20:00 UTC
          • 0 20 * * Fri
          • 0 20 * * 5
        • cada dues hores, a dos quarts / every 2 hours, at half past
          • 30 */2 * * *
        • a les 12 del migdia i a les dotze de la nit / at midnight and noon
          • 0 0,12 * * *
        • a les / at 1:30, 2:30, 3:30
          • 30 1-3 * * *
    • llistat / list
      • crontab -l
    • edició / edition
      • [export EDITOR=emacs]
      • crontab -e
    • logs
      • /var/log/cron
      • check mail of the owner of the crontab
    • Problemes / Problems
      • sudo: sorry, you must have a tty to run sudo
    • afegir línia des de shell / add line from shell:
      • How to create cronjob using bash
      • /var/spool/cron/crontabs/$USER
      • crontab -l | { cat; echo "0 0 * * * some entry"; } | sort -u | crontab -
      • root crontab:
        • sudo sh -c "crontab -l | { cat; echo '0 4 * * * some entry'; } | sort -u | crontab -"
    • Linux Execute Cron Job After System Reboot
      • @reboot
    • Crontab with Fabric
  • cu
  • curl
  • cut (esborra parts d'una línia / remove parts of a line)
    • get the second field when delimiter character is '_':
      • echo 'someletters_12345_moreleters.ext' | cut -d'_' -f 2
        12345
  • date
    • command
      		 output
      		 notes

      		LANG=en_GB
      		 LANG=en_US
      		 LANG=ca_ES.UTF-8
      date
      		 Mon 20 Feb 12:07:54 CET 2017
      		 Mon Feb 20 12:08:14 CET 2017
      		 dl feb 20 12:08:27 CET 2017
      		 see languages ()
      date '+%c'
      		 Mon 20 Feb 2017 12:06:06 CET
      		 Mon 20 Feb 2017 12:06:18 PM CET
      		 dl 20 feb 2017 12:00:12 CET see:
      date '+%x'
      		 20/02/17
      		 02/20/2017
      		 20/02/17
      date -Iseconds -u
      date --iso-8601=seconds --utc
      date '+%Y-%m-%dT%H:%M:%S%z' --utc       		2014-10-31T11:03:02+0000
      date -Ins --utc
      		 2016-06-30T11:51:20,083994321+0000
      date '+%Y-%m-%dT%H:%M:%S.%N%z' --utc 2016-06-30T11:51:20.083994321+0000 NOT valid for default date format in DRF
      date '+%Y-%m-%dT%H:%M:%S.%06NZ' --utc
      		 2016-06-30T13:57:26.562720Z
      		 valid for default date format in DRF
      date '+%Y-%m-%dT%H:%M:%SZ' --utc
      		 2015-11-10T16:28:56Z
      date '+%Y-%m-%dT%H%M%SZ' --utc
      		 2015-11-10T162945Z
      date '+%Y%m%d_%H%M' 20141031_1203
      date +%s 1414753430
      		 seconds
      date +%s%03N 1414753430123 milliseconds
      date +%s%06N 1414753430123456 microseonds
      date +%s%N 1414753430123456789 nanoseconds
      date -d '2012-11-20 19:19:00' +%s
      		1353435540
      		 seconds
      date --date=@1530780235

      		dj jul  5 10:43:55 CEST 2018
      		 seconds from 01/01/1970
      date --rfc-2822
      date -R
      		 Fri, 16 Sep 2016 09:34:17 +0200
      		 used by curl -z ... / -H 'If-Modified-Since: ...'
    • present date in ISO 8601 format (-I or --iso-8601), up to seconds precision (-Iseconds), UTC (-u)
      • date -Iseconds -u
    • present date in ISO 8601 format (-I or --iso-8601), up to nanoseconds precision (-Ins), UTC (-u)
      • date -Ins -u
    • present date in a given format:
      • date '+%Y%m%d_%H%M'
    • seconds between 1970-01-01 00:00:00 UTC and now (%s)
      • date +%s
    • milliseconds between 1970-01-01 00:00:00 UTC and now
      • date +%s%03N
    • seconds between a given date and 1970-01-01 00:00:00 UTC:
      • date -d '2012-11-20 19:19:00' +%s
    • Re: find out UTC in seconds since Jan 01 1900
      • date -d '2012-11-20 19:19:00' +2208988800+%s | bc
    • dateutils
      • ...
    • timedate addition
      • now=$(date -Iseconds -u)
        duration_seconds=10
        end_date=$(date '+%Y-%m-%dT%H:%M:%SZ' --date="$now + $duration_seconds seconds" -u)
      • now=$(date -Ins -u)
        duration_seconds=10.5
        end_date=$(date '+%Y-%m-%dT%H:%M:%S.%06NZ' --date="$now + $duration_seconds seconds" -u)
      • now_ms=$(date +%s%03N -u)
        duration_ms=300
        end_date_ms=$(( now_ms + duration_ms )        )
    • time addition
      • milliseconds
        • initial_time="00:01:00.100"
          delta_seconds="45.678"
          date -u -d "0 $(date -u -d "${initial_time}" +"%s.%03N") sec + $(date -u -d @"${delta_seconds}" +"%s.%03N") sec" +"%H:%M:%S.%03N"
    • time difference
      • max_time="09:22:01"
        min_time="08:44:55"
        date -u -d "0 $(date -u -d "$max_time" +"%s") sec - $(date -u -d "$min_time" +"%s") sec" +"%H:%M:%S"
      • microseconds precision:
        • max_time="09:22:01.123456"
          min_time="08:44:55.654321"
          date -u -d "0 $(date -u -d "$max_time" +"%s.%06N") sec - $(date -u -d "$min_time" +"%s.%06N") sec" +"%H:%M:%S.%06N"
      • date -u -d @$(( $(date -u -d "$max_time" +"%s") - $(date -u -d "$min_time" +"%s") )) +"%H:%M:%S"
    • datetime difference
      • How to calculate time difference in bash script?
        • $SECONDS
      • Find difference between two dates in bash
      • formatted output (integer precision)
        • min_date="2016-03-03T22:00:00Z"
          max_date="2016-03-04T22:30:00Z"
          date -d @$(( $(date -d "$max_date" +%s) - $(date -d "$min_date" +%s) )) -u +'%H:%M:%S'
      • output in seconds (integer precision)
        • min_date="2016-03-03T22:00:00Z"
          max_date="2016-03-04T22:30:00Z"
          #date -d @$(( $(date -d "$max_date" +%s) - $(date -d "$min_date" +%s) )) -u +'%s'
          delta=          $(( $(date -d "$max_date" +%s) - $(date -d "$min_date" +%s) ))
          echo $delta
      • output in seconds (nanosecond precision)
        • min_date="2016-03-03T22:00:00.123456789Z"
          max_date="2016-03-04T22:30:00.987654321Z"
          delta=$( echo $(date -d "$max_date" +%s.%N) - $(date -d "$min_date" +%s.%N) | bc -l )
          echo $delta
    • time comparison
      • time_1="13:58:08.200"
        time_2="15:58:08.200"

        if [[ ${time_1} < ${time_2} ]]
        then
            echo         "${time_1} < ${time_2}"
        else
            echo "${time_1} >= ${time_2}"
        fi
    • date comparison
      • date_1="2017-11-01T13:58:08.200Z"
        date_2="2017-11-01T14:58:08.000Z"

        if [[ $date_1 < $date_2 ]]
        then
            echo "$date_1 < $date_2"
        else
            echo "$date_1 >= $date_2"
        fi
    • convert seconds to time
      • time_s=10.5
        time=$(date -d@${time_s} -u +%H:%M:%S.%06N)
    • convert time (hh:mm:ss.dd) to seconds:
      • time="00:01:02.30"
        IFS=: read -r h m s <<<"$time"
        time_s=$( echo "($h * 60 + $m) * 60 + $s" | bc -l)
  • dd
    • Tuning dd block size
    • remove the first 35 bytes from a file:
      • dd bs=35 skip=1 if=full_file.bin of=stripped_file.bin
    • from a raw file (e.g. yuv) with lines of 1440 bytes, take the 100th and 400th lines:
      • dd bs=1440 skip=100 count=1 if=test.uyvy | od -v -t u1
      • dd bs=1440 skip=400 count=1 if=test.uyvy | od -v -t u1
    • image from an sd card to a file (e.g. from a Raspberry Pi)
      • dd bs=4M if=/dev/mmcblk0 | gzip > /tmp/raspbian.bin.gz
    • create a file of 1MB containing zeros:
      • dd if=/dev/zero of=1M_zeros.dat bs=1M count=1
    • create a file of 1MB containing random values:
      • dd if=/dev/urandom of=1M_random.dat bs=1M count=1
  • diff
  • dig
  • dmesg
  • dmidecode
    • Informació sobre el maquinari / Hardware information
  • du
    • Utilització de disc / Disk usage
      • du -cs * | sort -h
  • dump
    • od (octal dump)
    • xxd (hexadecimal dump)
  • echo
    • print variable content with a final \n:
      • echo $var
    • print variable content (preserve inner \n), with a final \n:
      • echo "$var"
    • do not print final \n
      • echo -n ...
    • eval escaped chars
      • echo -e ...
      • print character specified by its ascii value specified in octal:
        • old bash (<3.2):
          • echo -e \nnn...
        • new bash (>=3.2)
          • echo -e \0nnn...
  • eval
  • exec
  • expect
  • file
    • more info
      • file -b toto.file
    • mime type
      • file -b --mime-type toto.file
    • text coding
      • file -bi toto.tex
  • find
    • find
    • Find (UNIX Grymoire)
    • sintaxi / syntax:
      • opcions / options
      • proves / tests
        • -mtime +7 (més d'una setmana de temps de modificació / more than a week of modification time)
        • -newerXY
        • -type f (només fitxers / only files)
        • -perm +6000
        • -name ...
        • combinacions
          • -a -and
          • -o -or
          • !
      • accions / actions
    • -name
      • globular expressions (wildcard)
      • to use regular expressions instead of globular expressions, see regextype
      • regular expression
      • get all txt files, but exclude *bak*.txt
        • find . -name "*.txt" ! -name "*bak*"
      • remove all ts files or jpeg files older than 7 days
        • find . -type f \( -name "*.ts" -or -name "*.jpeg" \) -mtime +7 -delete
    • -regextype posix-extended -regex ...
      • How to use regex in file find
      • find all directories with name a-toto_..., b-toto_..., toto_... (but not c-toto_...)
        • find . -maxdepth 1 -type d -regextype posix-extended -regex '\./(a-|b-|)toto_.*'
    • -type
      • find regular files
        • find . -type f
      • find symbolic links (-type l)
        • find . -type l
      • find broken (-L) symbolic links (-type l)
        • find -L . -type l
      • find subdirectories
        • find . -maxdepth 1 -type d
    • -delete
      • delete files older than 5 days (print the list first):
        • find . -type f -mtime +5 -print0 | xargs -0 ls -ltr
        • find . -type f -mtime +5 -delete
      • delete all *.ts files (not beginning with a dot), without entering into the subdirectories (maxdepth 1), with less than 1k:
        • find . -maxdepth 1 -size -1k -name "[^\.]*.ts" -delete
      • recursively delete all symbolic links
        • find . -type l -delete
      • delete broken (-L) symbolic links (-type l)
        • find -L . -type l -maxdepth 1 -delete
        • verbose (-print)
          • find -L . -type l -maxdepth 1 -print -delete
      • recursively delete directories older than 150 days:
        • find . -maxdepth 1 -type d -mtime +150 -exec rm -rf {} \;
    • -exec executes command for each output item; a pipe to xargs takes the output as a single string
      • list (detailed) all files with extension m2v:
        • find . -name "*.m2v" -exec ls -l '{}' \;
        • find . -name "*.m2v" -ls
      • Copy all the files (-type f) in /var/export/backup_zeus3/ modified in the latest 24h (-mtime -1) to /tmp:
        • find /var/export/backup_zeus3/ -mtime -1 -type f -exec cp -pf '{}' /tmp \;
      • Recursively delete all directories named ".svn":
        • find . -name ".svn" -type d -exec rm -rf {} \;
      • Add ".bak" suffix to all txt files:
      • find . -name '*.txt' -exec mv {} {}.bak \;
    • | xargs
      • xargs by examples
      • list of found files (one line per file):
        • find . -name "*uyvy" -exec echo '{}' \;
      • list of found files (one line with all the files):
        • find . -name "*uyvy" -exec echo '{}' +
      • Calculates the total size of files modified today (-exec produces another behaviour):
        • find . -mtime -1 | xargs du -c
      • Fa un ls de cadascun dels fitxers (el nom pot incloure espais) / ls of each of the files (the file name can contain spaces):
        • find $PWD -name "*.html" -print0 | xargs -0 ls -l
      • Fa un zip dels fitxers */$trimestre/*.pdf
        • find . -name "*.pdf" -print0 | grep -FzZ ${trimestre} | xargs -0 zip ${trimestre}.zip
    • amb grep / with grep
      • grep toto $(find . -type f -newermt 2022-09-28)
      • find . -type f -newermt 2022-09-28 -exec grep toto '{}' \;
      • find . -type f -newermt 2022-09-28 | xargs grep tot
    • -print0
      • la separació dels resultats es fa amb '\0' en lloc d'espais / the separation of the results is made with '\0' instead of spaces
      • si va seguit per xargs, cal posar l'opció '-0' / if followed by xargs, xargs must have the option '-0'
    • -printf (%p: path and file name; %h: only path; %f: only file name)
      • find . -type f -name "*txt" -printf "fitxer: %h %f\n"
      • find . -type f -name "*txt" -printf "fitxer: %p\n"
    • How can I escape white space in a bash loop list?
    • Esborra amb ex...
    • Ignora alguns directoris / Ignore certain dirs
      • find . -name toto -not -path './not_here*'
    • dates
      • rsync with date filter
      • Unix/Linux find and sort by date modified
      • fitxers del 31 de març de 2013 / files modified on 31st March 2013:
        • find . -newermt 2013-03-31 ! -newermt 2013-04-1
      • fitxers entre el 9 d'agost de 2013 i l'11 d'agost de 2013 (ambdós inclosos), ordenats per temps:
        • find . -newermt 2013-08-9 ! -newermt 2013-08-12 -exec ls -ltr '{}' +;
        • find . -newermt 2013-08-9 ! -newermt 2013-08-12 -printf "%T@ %Tc %p\n" | sort -n
        • find . -newermt 2013-08-9 ! -newermt 2013-08-12 -printf "%T@ %p\n" | sort -n | awk '{print $2}'
        • find . -newermt 2013-08-9 ! -newermt 2013-08-12 -printf "%T@ %p\n" | sort -n | awk '{print $2}' | paste -s -d ' '
  • for
  • getent
    • getent [ahosts ahostsv4 ahostsv6 aliases ethers group gshadow hosts netgroup networks passwd protocols rpc services shadow]
  • grep
  • groups
  • head
    • see also tail
    • print first 2 lines
      • head -2
  • htop
  • iconv
  • id
  • if
  • ip
    • list all
      • ip a
      • ip addr show
    • get ip address
      • Consider: hostname --ip-address
    • route
      • ip route list
      • ip route replace default via 192.168.43.1 dev wlan0  metric 101
      • ip route del default via 192.168.43.1 dev wlan0  metric 303
      • ...
  • ipcalc
  • join
  • journalctl
  • kill
  • konsole
  • locate
  • ln
  • ls (*) (*)
    • ls -ltr (reverse time sorted)
    • ls -l 0225[1-5,7-9].MTS ("[]" means one character)
    • precise local time: (see also rsync)
      • ls --full-time
      • files
        • /etc/localtime
        • /usr/share/zoneinfo/Europe/Andorra
        • /usr/share/zoneinfo/Etc/UTC
    • precise UTC:
      • TZ="UTC" ls --full-time
    • one file per line (note that argument is number one)
      • ls -1
    • horizontal
      • ls -x
    • only dirs
      • ls -d */ | cut -f1 -d'/'
      • for i in $(ls -d ${root_dir}/*/); do echo $(basename $i); done
    • filename expansion
    • sort numbers as humans (1, 2, 10, 20):
      • ls -v
    • count number of files
    • ...
  • lsblk
  • minicom
  • mkdir
  • mktemp
    • create a temporary file:
      • file_name=$(mktemp)
    • create a temporary dir
      • dir_name=$(mktemp -d)
  • mount
    • mount NFS
    • mount some partition from an image file (e.g. an image file from an entire SD card)
      • Mounting a hard disk image including partitions using Linux
      • Mounting a raw partition file made with dd or dd_rescue in Linux
      • fdisk -l raspbian_20150731.bin
        • Disk raspbian_20150731.bin: 14,9 GiB, 15931539456 bytes, 31116288 sectors
          Units: sectors of 1 * 512 = 512 bytes
          Sector size (logical/physical): 512 bytes / 512 bytes
          I/O size (minimum/optimal): 512 bytes / 512 bytes
          Disklabel type: dos
          Disk identifier: 0x307439bc

          Device                 Boot    Start      End  Sectors   Size Id Type
          raspbian_20150731.bin1          2048  1683593  1681546 821,1M  e W95 FAT16 (LBA)
          raspbian_20150731.bin2       1687552 31050751 29363200    14G 85 Linux extended
          raspbian_20150731.bin3      31050752 31116287    65536    32M 83 Linux
          raspbian_20150731.bin5       1695744  1818623   122880    60M  c W95 FAT32 (LBA)
          raspbian_20150731.bin6       1826816 31049727 29222912    14G 83 Linux
      • parted raspbian_20150731.bin
        • GNU Parted 3.2
          Utilitzant /disc/rpi/raspbian_20150731.bin
          Welcome to GNU Parted! Type 'help' to view a list of commands.
          (parted) unit                                                            
          Unit?  [compact]? B                                                      
          (parted) print                                                           
          Model:  (file)
          Disk /disc/rpi/raspbian_20150731.bin: 15931539456B
          Sector size (logical/physical): 512B/512B
          Partition Table: msdos
          Disk Flags:

          Number  Start         End           Size          Type      File system  Flags
           1      1048576B      862000127B    860951552B    primary   fat32        lba
           2      864026624B    15897985023B  15033958400B  extended
           5      868220928B    931135487B    62914560B     logical   fat16        lba
           6      935329792B    15897460735B  14962130944B  logical   ext4
           3      15897985024B  15931539455B  33554432B     primary   ext4
      • Mount first ext4 partition (6)
        • mkdir /mnt/toto
        • mount -o loop,ro,offset=935329792 raspbian_20150731.bin /mnt/toto
      • resize partition in an image file
        • How To : Resize Partitions in an Image File
        • resize2fs
        • create a device (/dev/loop0) for all partitions:
          • losetup -f --show raspbian_20150731.bin
          • parted /dev/loop0
            • unit
              • B
            • print
              • Number  Start         End           Size          Type      File system  Flags
                 1      1048576B      862000127B    860951552B    primary   fat32        lba
                 2      864026624B    15897985023B  15033958400B  extended
                 5      868220928B    931135487B    62914560B     logical   fat16        lba
                 6      935329792B    15897460735B  14962130944B  logical   ext4
                 3      15897985024B  15931539455B  33554432B     primary   ext4
        • create a device (/dev/loop1) for partition 6:
          • losetup -f --show -o 935329792 raspbian_20150731.bin
          • parted /dev/loop1
            • unit
              • B
            • print
              • Number  Start  End           Size          File system  Flags
                 1      0B     14996209663B  14996209664B  ext4
        • check partition 6
          • e2fsck -f /dev/loop1
        • get the minimum size of partition 6
          • resize2fs -P /dev/loop1
            • resize2fs 1.42.12 (29-Aug-2014)
              Estimated minimum size of the filesystem: 1712943
            • as units are 4k, this means: 1712943*4096 = 7016214528 (~7GB)
        • resize partition 6
          • resize2fs /dev/loop1 8G
        • remove loops
          • losetup -d /dev/loop0 /dev/loop1
    • Problemes / Problems
      • SELinux
      • /var/log/messages: systemd: Unit mnt-vol1.mount is bound to inactive unit dev-xvdh.device. Stopping, too.
        • Solution:
          • systemctl daemon-reload
  • nc / ncat / netcat
  • netstat
  • nmap
  • nslookup
  • paste (posa línies en una sola / put lines into one single line)
    • put all ts files into one single line, where filenames are separated by a '|' (useful when concatenating files):
      • find . -name "*.ts" | paste -s -d '|'
    • put three consecutive lines into three columns:
      • paste -s -d '\t\t\n' input.dat > output.dat
    • other examples:
  • patch
  • pgrep
  • pkill
    • kill (SIGKILL) all child processes
      • pkill -KILL -P $$
  • printf
    • The printf command
    • integer format with zeros:
      • printf "%06d" $my_var
    • thousand separator (according to locale)
      • printf "%'d" ${my_var}
    • string and numbers with padding spaces (minus sign for left-aligned)
      • printf "%-40s%20d\n" ${my_string} ${my_int}
    • quote special characters
      • printf "%q" $my_var
    • Colors / Colours
      • Standard ECMA-48: Control Functions for Coded Character Sets
      • C
      • start colouring
        • "\e[<attributes>m"
        • Python: "\033[<attributes>m"
      • attributes are ";" separated
        • 0
          		 all attributes off
          1
          		 bold on
          2 dark on
          4
          		 underscore
          5
          		 blink on
          54 upperscore
          7
          		 reverse video on
          8
          		 concealed on
          foreground colour
          		 background
          30
          		 black 40
          31
          		 red 41
          32
          		 green 42
          33
          		 yellow 43
          34
          		 blue 44
          35
          		 magenta 45
          36
          		 cyan 46
          37
          		 light grey 47
          90 dark grey 100
          91 light red 101
          92 light green 102
          93 light yellow 103
          94 light blue 104
          95 light magenta 105
          96 light cyan 106
          97 white 107
      • end colouring
        • "\e[0m"
        • Python: "\33[0m"
      • Exemples / Examples

        		• shell Python
          print "Title" in green foreground (32) "\e[32mTitle\e[0m" "\33[32mTitle\33[0m"
          print "my_command" in blue foreground (34) "\e[34mmy_command\e[0m" "\33[34mmy_command\33[0m"
          print "my_outout" in yellow foreground (33) "\e[33mmy_output\e[0m" "\33[33mmy_output\33[0m"
          Print "Hello", in blue foreground (34),
          yellow background (43), blinking (5)          		 "\e[5;34;43mHello\e[0m\n"
      • emacs
  • ps
  • pstree
    • CentOS
      • sudo yum install psmisc
  • pv
  • read
    • read one line from stdin and populate specified variables
    • read <options> <list of vars>
    • options
      • option
        		 description
        		 examples
        -r
        		 The backslash is considered to be part of the line, and not interpreted as an escape character
        -s
        		 silent
        -t <secs>
        		 timeout



    • used variables
      • variable
        		 description
        		 examples
        		 info
        IFS
        		 Internal Field Separator
        		 IFS= (or IFS='')  prevents leading/trailing whitespace from being trimmed
    • line by line from a file
    • read from a file, containing three fields (separated by space) per line
      • while read -r field1 field2 field3
        do
            echo $field1 $field2 $field3
        done < $filename
    • Bucle sobre json amb jq / Loop over json from jq
    • iterate over lines that can contain spaces (a simple for would only work if there were no spaces in lines) from a variable:
      • #! /bin/bash

        fitxers_html=$(find $PWD/ -name "*.html")

        while IFS= read -r linia
        do
            echo "${linia}"
            sed -i 's#src="http://www.w3.org/Icons/valid-html401"#src="//www.w3.org/Icons/valid-html401"#g' "${linia}"
        done < <(echo "$fitxers_html")

        exit 0
      • #!/bin/bash

        tags=$(cat <<EOF
        TAG:major_brand=mp42
        TAG:minor_version=0
        TAG:compatible_brands=isommp42
        TAG:creation_time=2016-10-27 14:41:05
        EOF
        )
        echo "tags: $tags"

        while IFS= read -r
        do
            tag=$REPLY
            echo "tag: $tag"
            pair=${tag#TAG:}
            echo "pair: $pair"
        done < <(echo "$tags")
  • recode
  • rm
    • esborra el fitxer "-nom_fitxer":
      • rm -- -nom_fitxer
    • esborra recursivament amb excepcions / recursively remove with exceptions:
      • find dir -maxdepth 1 -mindepth 1 ! -name "name_to_avoid" -exec rm -rf {} \;
    • esborra els fitxers el nom dels quals està especificat en un fitxer:
      • ls *.mp4 -1 > mp4_files_to_be_deleted.txt
      • xargs rm < mp4_files_to_be_deleted.txt
    • esborra els fitxers apuntats per enllaços simbòlics
  • rsync
  • screen
    • serial
    • Screen Command: Set Baud Rate [ Terminal Communication ]
    • Commands
      • Common screen commands
      • screen /dev/ttySX baud_rate,cs8|cs7,ixon|-ixon,ixoff|-ixoff,istrip|-istrip
        • baud_rate
        • cs7|cs8: 7|8 bits per byte
        • ixon|-ixon: enable|disable software flow-control (CTRL-S/CTRL-Q) for sending data
        • ixoff|-ixoff: enable|disable flow-control (CTRL-S/CTRL-Q) for receiving data
        • istrip|-istrip: clear|keep the eight bit in each received byte
      • CTRL+a ?
        		 help
        CTRL+a i
        		 info
        CTRL+a K
        		 kill session
        ...
  • sed
    • Sed - An Introduction and Tutorial by Bruce Barnett
    • Unix Sed Tutorial: Append, Insert, Replace, and Count File Lines
    • sed [-n] [-e] '[/begin_pattern/[,/end_pattern/]] [!]command' [-e 'command'] ... input_filename
    • sed [-n] [-e] '[/begin_pattern/[,/end_pattern/]] [!]{command}' [-e 'command'] ... input_filename
    • Command line options
      • -f filename
        		 execute script in a file (several commands can be specified, each in one line)
        -e script
        		 execute script (no need to specify -e for the first script)
        -n only print the matching pattern
        -i replace input file
        ...
    • Commands
    • Inserció de línies / Line insertion
      • before
        • sed '/This is an existing line/ i This is a new line' toto.txt
        • spaces has to be escaped:
          • sed -i '/<link rel="stylesheet" href="css\/fpm.css"/ i\  <meta name="viewport" content="width=device-width">' toto.html
      • after
        • sed '/This is an existing line/ a This is a new line\n and a second one' toto.txt
    • Substitució / Replacement
      • Per a substituir totes (g) les cadenes inici (pot ser una expressió regular) en el fitxer fitxer_original per final, en fitxer_final:
        • sed 's/inici/final/g' fitxer_original > fitxer_final
        • per a sobreescriure el fitxer inicial: "-i" / to overwrite original file
          • sed -i 's/inici/final/g' fitxer_original
          • create a backup (fitxer_original.bak)
            • sed -i.bak 's/inici/final/g' fitxer_original
        • el separador no cal que sigui '/'. Pot ser qualsevol altre caràcter.
        • si a dins de l'ordre volem fer servir variables, cal posar-la entre cometes dobles:
          • baseurl=$1 sed -i "s#<BaseURL>.*</BaseURL>#<BaseURL>${baseurl}</BaseURL>#g" toto.html
      • newline
        • sed: How can I replace a newline (\n)?
        • remove line feed (^M) (see also dos2unix):
          • sed -i 's/\r//g' toto.txt
        • replace all "/n" (2 chars) by real newlines in a variable:
          • echo "$toto" | sed 's/\\n/\n/g'
      • Replace ' -> '\'' (e.g. for curl POST with a specified json)
        • echo "${input}" | sed "s/'/'\\\''/g"
      • Estableix BaseURL en un fitxer mpd (MPEG DASH):
        • #! /bin/bash
          baseurl=$1
          fitxers_mpd=$(find $PWD -name "*.mpd")
          for fitxer in $fitxers_mpd; do
              echo "${fitxer}: setting BaseURL to ${baseurl}"
              sed -i "s#<BaseURL>.*</BaseURL>#<BaseURL>${baseurl}</BaseURL>#g" $fitxer
          done
      • En tots els fitxers del directori public_html:
        • #! /bin/bash
          for f in public_html/*
          do
              echo $f;
              sed -i.bak 's*<P ALIGN=center>*<P STYLE="text-align:center">*g' $f;
          done
        • #! /bin/bash
          for f in public_html/*;
          do
              echo $f;
              sed 's*<P ALIGN=center>*<P STYLE="text-align:center">*g' $f > ${f}#;
              rm -f $f;
              mv ${f}# $f;
          done
      • Canvia la mida de la lletra dels textos en Android / Change the font size in Android texts:
        • find . -name *.xml -exec sed -i.bak 's/android:textSize="10sp"/android:textSize="12sp"/g' {} \;
      • Afegeix un meta refresh a tots els fitxers html:
        • #! /bin/bash
          fitxers_html=$(find $PWD -name "*.html")
          for fitxer in $fitxers_html;
          do
              echo "${fitxer}"
              fitxer_nom=$(basename ${fitxer})
              inicial="<head>"
              final="<head>\n  <meta http-equiv=\"refresh\" content=\"5; url=http://www.francescpinyol.cat/${fitxer_nom}\">"
              sed -i "s#${inicial}#${final}#g" ${fitxer}
          done
      • Comenta totes les línies que comencin per "net.ipv4.ip_forward":
      • Uncomment all lines that have an "=":
        • sed -e '/=/ s/^#//' -i manila.rpm.uncommented.conf
      • Uncomment lines that start with #submission and subsequent lines with options, unil the next line that is not an option (in the example, line starting with smtps):
        • /etc/postfix/master.cf
          • ...
            #submission inet n       -       n       -       -       smtpd
            #  -o syslog_name=postfix/submission
            #  -o smtpd_tls_security_level=encrypt
            #  -o smtpd_sasl_auth_enable=yes
            #  -o smtpd_reject_unlisted_recipient=no
            #  -o smtpd_client_restrictions=$mua_client_restrictions
            #  -o smtpd_helo_restrictions=$mua_helo_restrictions
            #  -o smtpd_sender_restrictions=$mua_sender_restrictions
            #  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
            #  -o milter_macro_daemon_name=ORIGINATING
            #smtps     inet  n       -       n       -       -       smtpd
            ...
        • activate_submission.sed
          • /^#submission/,/^#\w/ {
                /^#submission/ s/^#//
                /^#  / s/^#//
            }
        • sed -f activate_submission.sed master.cf
          • ...
            submission inet n       -       n       -       -       smtpd
              -o syslog_name=postfix/submission
              -o smtpd_tls_security_level=encrypt
              -o smtpd_sasl_auth_enable=yes
              -o smtpd_reject_unlisted_recipient=no
              -o smtpd_client_restrictions=$mua_client_restrictions
              -o smtpd_helo_restrictions=$mua_helo_restrictions
              -o smtpd_sender_restrictions=$mua_sender_restrictions
              -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
              -o milter_macro_daemon_name=ORIGINATING
            #smtps     inet  n       -       n       -       -       smtpd
            ...
    • Substitueix tota la línia / Replace all line:
      • sed -e "/^.\?admin_token/ c\admin_token=$ADMIN_TOKEN" -i /etc/keystone/keystone.conf
      • sed -e "/^host    all             all             127.0.0.1/ c\host    all             all             127.0.0.1/32            md5" /var/lib/pgsql/data/pg_hba.conf
      • canvia el valor de la variable toto_var:
        • sed '/var toto_var*/ c\var toto_var = "nou_valor"' toto.js
    • grep-like
      • sed -n /pattern/p file.txt
        • print only lines starting with "admin_token" or one single character followed by "admin_token":
          • sed -n '/^.\?admin_token/p' /etc/keystone/keystone.conf
        • print only line with "duration=" in it, and replace it by nothing (i.e. get the value) (e.g. to parse result given by ffprobe):
          • sed -n 's/duration=//p'
      • printing all lines except (!) those that match the pattern (grep -v):
        • sed -n -e '/pattern/ !p' file.txt
    • Ranges by patterns
    • elimina totes les línies que comencin per «abc»:
  • seq
    • similar to {10..20}, but with seq one of the limits can be a variable
    • END=20
      for i in $(seq 10 $END)
      do
          echo $i
      done
  • set
    • activate debug
      • set -x
    • set positional parameters
      • set -- param1 param2
        • Exemple / Example
          • #!/bin/bash

            echo "supplied $# parameters: $@"
            # add a parameter
            set -- "new_param1" "new_param2"
            echo "new $# parameters: $@"

            exit 0
  • shopt
  • sleep
  • sort
    • How to sort on Multiple Fields with different field separator
    • sort by months in English:
      • LC_ALL=en_US.utf8 sort ... -k2M ...
        • ...
      • cat logs_per_ordenar.txt | awk -F '[ /]' '{print $3"\t"$2"\t"$1"\t"$0}' | LC_ALL=en_US.utf8 sort -k1n -k2M | cut -f4-
        • input:
          • [15/Feb/2024 09:13:04] ...
            [17/Jan/2024 09:13:04] ...
            [17/Jan/2024 09:58:13] ...
            [17/Jan/2024 10:36:23] ...
            [16/Feb/2024 09:13:04] ...
            [17/Jan/2024 10:36:23] ...
            [17/Jan/2024 11:04:25] ...
            [02/Jan/2024 11:16:45] ...
            [17/Jan/2024 11:16:50] ...
            [17/Jan/2024 11:24:56] ...
            [30/Nov/2023 08:53:01] ...
            [30/Nov/2023 08:54:13] ...
            [02/Jan/2023 08:54:13] ...
            [14/Feb/2024 09:13:04] ...
        • output:
          • [02/Jan/2023 08:54:13] ...
            [30/Nov/2023 08:53:01] ...
            [30/Nov/2023 08:54:13] ...
            [02/Jan/2024 11:16:45] ...
            [17/Jan/2024 09:13:04] ...
            [17/Jan/2024 09:58:13] ...
            [17/Jan/2024 10:36:23] ...
            [17/Jan/2024 10:36:23] ...
            [17/Jan/2024 11:04:25] ...
            [17/Jan/2024 11:16:50] ...
            [17/Jan/2024 11:24:56] ...
            [14/Feb/2024 09:13:04] ...
            [15/Feb/2024 09:13:04] ...
            [16/Feb/2024 09:13:04] ...
  • sponge
    • Instal·lació / Installation
      • Alma 8
        • sudo dnf install moreutils
    • Ús / Usage
      • sudo sh -c 'sort -u /etc/exports | sponge /etc/exports'
    • ...
  • ssh
  • su / sudo
    • RootSudo (Ubuntu)
    • sudoers file
      • su root
      • adduser mynewuser
      • passwd mynewuser
      • [su]
      • option 1: add files to /etc/sudoers.d (because they are included by the last line in /etc/sudoers file)
        • su
        • echo "mynewuser ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/mynewuser
      • option 2: directly edit sudoers file
        • visudo (don't use regular editor, as you can end up with a non-working system)
          • got to after line with "root ALL=(ALL) ALL"
          • i
          • mynewuser ALL=(ALL) ALL
            • to avoid password request every time you execute sudo, append the following line at the end of the file (/etc/sudoers):
              • mynewuser ALL=(ALL) NOPASSWD: ALL
          • ESC
          • :wq
      • Problemes / Problems
    • How to Add a User and Grant Root Privileges on CentOS 7
    • give a user admin privileges:
      • sudo adduser <username> admin
    • redirect
      • sudo sh -c 'echo "nameserver 192.168.0.1" >> /etc/resolv.conf'
    • execute as another user (which has home)
      • su my_user -c "mkdir /mnt/nfs/my_dir"
      • sudo su - my_user -c "mkdir /mnt/nfs/my_dir"
    • execute as another user (which has no home; created as --no-create-home)
      • su nginx -s /bin/bash -c "mkdir /mnt/nfs/my_dir"
      • sudo -u nginx mkdir /mnt/nfs/my_dir
      • sudo su - nginx -s /bin/bash -c "export"
    • mode interactiu / interactive mode
      • sudo -i
    • mode gràfic / graphical mode
      • Gnome
        • gksudo gedit /etc/fstab
      • KDE
        • kdesudo kate /etc/X11/xorg.conf
      • LXDE
        • gksudo leafpad /etc/fstab (text editor)
        • gksudo pcmanfm /etc (file browser)
    • sudo in ssh, crontab
      • to avoid having to specify -t option (to avoid error: sudo: sorry, you must have a tty to run sudo):
        • /etc/sudoers.d/80-my_user
          • myusername ALL=(ALL) NOPASSWD: ALL
            # no need to specify "-t" when doing sudo via ssh for this user
            Defaults:myusername !requiretty
        • for all users: /etc/sudoers
          • Defaults !requiretty
    • while in ssh
      • ssh -q remote_user@remote_server -t "sudo bash -c \"while ! [ -e /dev/xvdf ]; do sleep 1; done\""
  • sysctl
  • tail
    • see also: head
    • get latest 5 lines
      • tail -n5
  • tar
    • root tarball needs to be extracted with --numeric-owner
    • ignore git dirs:
      • tar --exclude=.git --exclude=.gitignore -cvzf toto.tgz my_dir
    • follow symbolic links (-h)
      • tar chvf documents.tgz Documents
    • uncompress
      • extension
        		 tar
        .tgz, .tar.gz
        		 -xzf
        .bz2
        		 -xjf
        .xz
        		 -xJf
    • partially remove path when creating (/mnt/vol1 will not appear in paths for files inside the tar file):
      • tar cvzf toto.tgz --directory /mnt/vol1 letsencrypt
  • tee
  • telnet
  • test
  • timeout
  • tip
  • top
  • tr
    • substitució de caràcters individuals / replacement of individual characters:
      • tr '<input_characters>' '<output_characters>'
    • elimina les comes i cometes / delete comma and quotes:
      • tr -d '",'
    • elimina les cometes simples / delete single quotes
      • tr -d "'"
    • remove line feed (^M):
      • tr -d '\r'
    • majúscules i minúscules / upper case and lower case
      • ...
    • url safe
  • trap
    • indica una funció que cal executar quan es rep el senyal, excepte KILL (9), en lloc del comportament habitual
    • How "Exit Traps" Can Make Your Bash Scripts Way More Robust And Reliable
      • Capping Expensive Resources: AWS AMI after updating an instance
    • Sending and Trapping Signals
    • Identifying received signal name in Bash
    • Examples
      • trap <code or function> <list of signals>
      • #!/bin/bash

        trap "rm $TEMP_FILE; exit" SIGHUP SIGINT SIGTERM
        ...
      • #!/bin/bash

        function on_exit {   
            # variable to know whether we arrive here from an "exit 0" or "exit 1"
            rv=$?

            # do something before exiting (e.g. remove temporary files)
            ...
        }

        # when EXIT signal is sent to this script (either from an exit command or from a CTRL-C), go to finish function
        trap on_exit EXIT

        ...

        if ...
        then
           exit 1
        fi

        exit 0
      • #!/bin/bash

        function on_exit {   
            # variable to know whether we arrive here from an "exit 0" or "exit 1"
            rv=$?

            # do something before exiting (e.g. kill all children processes)
            own_pid=$$
            echo "sending TERM to all children processes (whose parent id is ${own_pid}):"
            # only informative:
            # install pstree with: sudo yum install psmisc
            command -v pstree >/dev/null 2>&1 && pstree -p ${own_pid}
            # install pgrep with: sudo yum install procps
            command -v pgrep >/dev/null 2>&1 && pgrep -a -P ${own_pid}

            pkill -TERM -P $$
        }

        # when EXIT signal is sent to this script (either from an exit command or from a CTRL-C), go to finish function
        trap on_exit EXIT

        ...

        if ...
        then
           exit 1
        fi

        exit 0
  • tree
    • add colours:
      • tree -C .
  • umask
  • useradd
    • with specified password:
      • useradd --password $(openssl passwd -1 ${new_password}) ${new_user}
  • usermod
  • wait
  • which
  • while
  • xargs
    • takes input and passes it as arguments to the specified command. Input can be separated by whitespaces (default) or by '\0' (option: -0).
    • find and xargs
    • remove files specified in a file as one file per line
    • list files specified in a file as one file per line
      • xargs ls -l < list_files.txt
      • cat list_files.txt | grep something | xargs ls -s
  • zip / unzip
    • compress
      • recursive
        • zip toto.zip -r my_dir
      • encrypt with a password (prompt):
        • zip -e toto.zip -r my_dir
      • with find
        • #!/bin/bash

          EXPECTED_ARGS=1
          if [ $# -ne $EXPECTED_ARGS ]
          then
              cat <<EOF
          Usage: `basename $0` yyyy_t
          EOF
              exit 1
          fi

          PREFIX=my_prefix
          trimestre=$1
          find . -name "*.pdf" -print0 | grep -FzZ ${trimestre} | xargs -0 zip ${PREFIX}_${trimestre}.zip
    • uncompress
      • unzip -d /destination/dir toto.zip
    • password protection

Usuaris i grups / Users and groups
  • Usuaris / Users:
    • A Complete Guide to Usage of ‘usermod’ command – 15 Practical Examples with Screenshots
    • Llista d'usuaris / List of users
    • Crea un usuari / Create a user:
      • useradd user_name -u user_id -g main_group -G additional_group_1,additional_group_2
      • sense casa / with no home:
        • adduser --system --no-create-home --user-group -s /sbin/nologin nginx
        • make a dir as user with no login
        • add path to PATH variable for all users (including those without home)
          • /etc/profile.d/local_path.sh
            • # add /usr/local/bin path for all users
              pathmunge /usr/local/bin
    • Afegeix contrasenya a un usuari / Add password to a user:
    • Afegeix grups addicionals a un usuari / Add additional groups to a user:
      • usermod -a -G additional_group_1,additional_group_2 user_name
        • Nota: "-a" afegeix els grups secundaris citats. Si no, els substitueix per la nova llista
      • usermod -a -G dialout my_user
    • Elimina tots els grups secundaris d'un usuari:
      • usermod -G "" user_name
    • Llista les propietats i grups d'un usuari:
      • id nom_usuari
    • ID d'usuari de l'usuari actual:
      • id -u
    • ID de grup de l'usuari actual:
      • id -g
    • Llista de groups dels quals l'usuari actual és membre:
      • groups
    • Canvia l'uid de l'usuari (fa els canvis pertinents als fitxers del directori arrel de l'usuari) / Change user id:
      • # killall -u nom_usuari
      • # usermod -u nou_uid nom_usuari
    • Actualització dels grups per a l'usuari actual, sense haver de reiniciar la sessió (the group has already been assigned to the user, and it is shown with id user_name):
      • newgrp additional_group_1
    • Esborra un usuari / Delete a user
      • userdel user_name
    • Sticky group / setgid (els nous fitxers i directoris a dins es crearan amb el grup del directori): 
    • umask
      • umask (wp)
      • mostra la màscara actual / show present mask
        • umask [-S]
      • estableix la màscara / set the mask:
        • umask txyz
        • permanentment / permanently
          • ~/.bashrc
            • umask 022
          • /etc/bashrc
            • umask 022
          • /etc/login.defs
            • UMASK 022
          • /etc/csh.login
            • umask 022
      • usage in:
    • Problemes / Problems
      • pam_succeed_if(su:auth): requirement "uid >= 1000" not met by user ...
    • add user to Samba
  • Grups / Groups
    • Howto: Linux Add User To Group
    • Llista de grups / List of groups
    • Crea un grup / Create a group:
      • groupadd [-r] -g group_id group_name
    • Llista els usuaris d'un grup / List the members of a group:
      • grep ^group_name /etc/group
    • Llista de grups d'un usuari
      • groups nom_usuari
    • Afegeix usuaris a un grup
    • Esborra usuaris d'un grup
    • Canvia el gid d'un grup / Change the gid of an existing group:
      • groupmod --gid <new_gid> <group_name>
    • chgrp additional_group_1 -R dir_name_1
    • chmod +s dir_name_1

Exemples / Examples
  • Detecció de fitxers duplicats, encara que tinguin noms diferents:
    • #!/bin/bash
      if [ $# -ne 2 ]
      then
          echo "Usage: " $0 dir1 dir2
      else
          dir1=$1
          dir2=$2
          echo "dir1: " $dir1
          echo "dir2: " $dir2

          for f1 in $dir1/*
          do
          if [ -f $f1 ]
          then
              echo "-> " $f1
              for f2 in $dir2/*
              do
              if [ -f $f2 ]
              then
                  if cmp $f1 $f2 > /dev/null 2>&1
                  then
                  echo "    " $f2 "iguals"
                  fi
              fi
              done
          fi
          done

      fi
  • Comprova si totes les imatges de la targeta SD han estat passades a un altre directori / Check if all the pictures in SD card have been copied to a local directory (comprova_fotos.sh):
    • #!/bin/bash

      EXPECTED_ARGS=2
      if [ $# -ne $EXPECTED_ARGS ]
      then
          cat <<EOF
      Usage: `basename $0` dir1 dir2

      E.g.: `basename $0` /run/media/cesc/41AD-7342/DCIM/101CANON/ ~/Imatges/Canon/
      EOF
          exit 1
      fi

      dir1=$1
      dir2=$2

      for fitxer_dir1 in ${dir1}/*
      do   
          if [ -f $fitxer_dir1 ]
          then
            fitxer1=$(basename $fitxer_dir1)
            trobat=$(find $dir2 -name $fitxer1)
            if [ -n "$trobat" ]
            then
              # 32: verd
              printf "\33[32m"
              printf "$fitxer1 -> $trobat"
            else
              # 31: vermell
              printf "\33[31m"
              printf "$fitxer1"
            fi
            # sense colors
            printf "\33[0m"
            printf "\n"
          fi

      done
  • Check mime-type (or use mimetype command instead of file command)
    • if [ `file -b --mime-type $1` = "text/plain" ]
      then
          echo "plain text"
      else
          echo "no plain text"
      fi
    • # any image mime-type
      if [[ $(file -b --mime-type $filename) =~ ^"image" ]]
      ...
  • Reading files
  • Subsample image file names (e.g. to make a faster timelapse from images)
    • #!/bin/bash

      EXPECTED_ARGS=1
      if [ $# -ne $EXPECTED_ARGS ]
      then
          cat <<EOF
      Usage: `basename $0` delta
      EOF
          exit 1
      fi

      delta=$1

      # remove all symbolic links
      find . -type l -delete

      counter=0
      for filename in *
      do
          # only image files
          if [[ $(file -b --mime-type $filename) =~ ^"image" ]]
          then
              echo $filename
              # file extension
              file_ext=${filename##*.}
              # only digits
              file_index=${filename//[^0-9]/}
              # remove leading zeros by converting to int in base 10
              file_index_wo_zeros=$((10#$file_index))
              # calculate modulo delta
              sub=$(( file_index_wo_zeros % delta ))
             
              if (( sub == 0 ))
              then
                  new_filename=$(printf "im_%05d.$file_ext" $counter)
                  echo "$filename -> $new_filename"
                  ln -s $filename $new_filename
                  (( counter++ ))
              fi
          fi
      done

      echo "You can play images with, e.g.: ffplay -i im_%05d.$file_ext"

http://www.francescpinyol.cat/shell.html
Primera versió: / First version: 4.III.2016
Darrera modificació: 28 de setembre de 2024 / Last update: 28th September 2024

Valid HTML 4.01!

Cap a casa / Back home