There are several ways of calling shell command in Python scripts, and very often I found that when calling shell commands, it will not finish running before it begin to run Python codes next to the shell command.
After many tries, the following codes can call shell command successfully sometimes (but not all the time):
process = subprocess.Popen(cmd,
stdout, stderr = process.communicate()
return_code = process.returncode
if return_code < 0:
raise Exception("fail to run "+str(cmd))
An example of calling this command is:
execute_command('apt install ffmpeg')
Here we also use time.sleep to make sure that the command can finish before running next lines.
make_subplots takes the following steps:
(1) generate a figure via
(2) add trace via
update_layout is used to reset the image setting
specsis an important parameter in
make_subplots, and it will determine the type of plots we want to have, for example, if we want to have
pieplot, then we should write
subplot_titlesis used to denote the title of each subplots
shared_xaxes=False; then use
fig.update_xaxesto set individual xaxes titles.
if only folders/sub-folders
(1) global search including sub-directories
my_list = pathlib.Path(my_dir).rglob("*.png")
(2) only search in current directory
my_list = pathlib.Path(my_dir).glob("*.png")
from pathlib import Path
my_path = Path.cwd() / 'python' / 'libpath' / "a.txt"
assert my_path.exists(), print(my_path)
.with_suffix : change the suffix name
cur_file_dir = os.path.dirname(__file__)
hydra_setting_path = os.path.relpath(hydra_setting_path, cur_file_dir)
If two variables refer to immutable objects that have equal values (a == b is True), in practice it rarely matters if they refer to copies or are aliases referring to the same object because the value of an immutable object does not change, with one exception. The exception is immutable collections such as tuples and frozensets: if an immutable collection holds references to mutable items, then its value may actually change when the value of a mutable item changes. In practice, this scenario is not so common. …
In order to represent data in Python, we have the following options:
We recommend to use dataclass as it provides a lot of possibilities.
However, using dataclass can be problematic from the software engineering perspective:
These are classes that have fields, getting and setting methods for fields, and nothing else.Such classes are dumb data holders and are often being manipulated in far too much detail by other classes.
— from Refactoring of Martin Fowler and Kent Beck
The main idea of…
In this blog I keep track of what I have configured my new Ubuntu machine for machine learning projects.
(1) Operating system
Ubuntu 18.04 LTS
(2) Settings from previous Ubuntu machine
a) check the GPU information by using
`sudo lshw -c video`, and you will know what kind of GPU you have in your PC.
`sudo ubuntu-drivers devices` find the GPU driver version that is recommended
c) install the recommended version
`sudo apt install nvidia-driver-version-number` or use
`sudo ubuntu-drivers autoinstall`, letting the computer choose the recommended GPU driver version for you. …
It may happen that sometimes we may need to install Ubuntu packages without Internet connection. Such a scenario happens to me one day when my desktop needs to compile the wireless adapter driver program using
make command before it can connect with the Internet. However, the native Ubuntu 18.04 system does not contain
build-essential package. So I have to find a way to install it offline. In this article, I record the way how I successfully install this Ubuntu package.
Step 1: in a Ubuntu machine that can connect with Internet, download the docker Ubuntu system that you have installed…
In the past few years, I have been using Markdown to write technical notes and then I use Mkdocs to organize them. Here is a summary of my experience.
pip install mkdocs-jupytermakes mkdocs supports jupyter-notebook
pip install mkdocs-materialand
pip install mkdocstringsmake mkdocs support Python code documentation
├── docs │…