System design
System design
- Whenever sudo is required, it’s prompted with the reason
- Log with different log levels are available, by default it’s enabled with debug
- Error handing with fail scenarios for missing dependency checks and recommendations
- different level of granularity of system setup
- Docker container compliance through modular configurations and environment variables
- Nvidia stack - driver, cuda, cudnn and tensorrt support for the host and docker containers
- Handpicked and tested for different version dependencies
- Tested on Ubuntu 18.04 LTS
Core Configuration Modules: lscripts/config
WARNING:
- DO NOT change the configuration variable names and imports order
- They are sensitive to the order and sequence in which they are defined, hence changing them will have unexpected consequences
- Be Aware and open the script in editor to understand what it is doing, specifically to dangerous function call that can purge or delete things; also better make them internal functions.
The reason for sensitivity towards order and sequence of definitions and imports is because I created a Cascading Pattern in order to pass variables & their respective values between different scripts without leaking them in the global scope or to terminal. There is no simple way to use a shell script as a library module, hence the decision is to focus on flexibility over rigidity, simplicity over complexity and modularity of shell scripts. The success of cascading pattern to variable passing is due to the unique name space creation mechanism through consistent naming conventions and namespace scoping.
- Uses shell scripts itself for creating configuration variables and files are named as
<configname>-cfg.sh
- All variables are in local scope, configuration files cannot be used directly and can be used inside functions only
- All variable names are uppercase
- Color codes configurations
_color_.sh
=>config/_color_.sh
- Internal usage for dynamic configurations
_typeformats_.sh
=>config/_typeformats_.sh
- Wraps all configurations and it’s the single entry point:
__init__.sh
=>config/__init__.sh
like:- Core configuration
- system
- basepath
- versions
- nvidia
- docker
- python
- Users configuration
- MongoDB configuration
- Docker container configuration
- Core configuration
Core Modules: lscripts/core
All the Code modules are loaded from: => lscipts/core/__init__.sh
. Here they are mentioned in the same order in which they are imported, their import orders should not be changed. There are two types of modules:
- wrapper modules - they only prints configuration currently
- functional modules - they additionally adds more functionalities
- log
- log module
lscripts/core/_log_.sh
- log module
- utils module
_utils_.sh
=>lscripts/core/_utils_.sh
- date module
_date_.sh
=>lscripts/core/_date_.sh
- color wrapper module
_color_.sh
=>lscripts/core/_color_.sh
- typeformats wrapper module
_typeformats_.sh
=>lscripts/core/_typeformats_.sh
- system module
_system_.sh
=>lscripts/core/_system_.sh
- file I/O module
_fio_.sh
=>lscripts/core/_fio_.sh
- Stack wrapper module
_stack_.sh
=>lscripts/core/_stack_.sh
- dir module
_dir_.sh
=>lscripts/core/_dir_.sh
- apt module
_apt_.sh
=>lscripts/core/_apt_.sh
- nvidia gpu and cuda stack module
_nvidia_.sh
=>lscripts/core/_nvidia_.sh
- docker module
_docker_.sh
=>lscripts/core/_docker_.sh
- mongodb wrapper module
_mongodb_.sh
=>lscripts/core/_mongodb_.sh
Common Module
- Common:
_common_.sh
:- high level wrapper
- wraps the code configurations and core functions
Naming conventions
- Variable names:
- namespace
LSCRIPTS
: suggested user defined environment variable namespace- use this as prefix for custom environment variables
- or use this as a variable names withing scripts when using
lscripts
as library module- as a convention, use uppercase is they are coming from environment variables, constant values or non-function global variables, otherwise use lowercase for variables in local scope, function names
LSCRIPTS__
: Environment variables namespace prefix<modulename>.<function_name>
: all module functions follow this pattern *_<SOME_NAME>
i.e. starting with single_
underscore- these are reserved variable names
- function names are tucked under module name eg:
lsd-mod.log.debug
where module name is_log_.sh
*__<SOME_NAME>
i.e. starting with double__
underscores - these are reserved variable names
- strictly private scope, overriding these has unexpected impact
- function names are tucked under module name.
- These should not be invoked directly and instead their wrapper function to be used eg:
lsd-mod.log.__failure
where module name is_log_.sh
is a expected to private, so instead use it’s wrapper:lsd-mod.log.fail
* Environment variables
- These should not be invoked directly and instead their wrapper function to be used eg:
- All environment variables that are expected to be customized by user defined values:
- namespace prefix:
LSCRIPTS__
- namespace prefix: