Python Programming Related

This page hosts general information about python programming. For detailed packages, check the deligated pages. For example, Tensorflow Related.

We use conda to manage package and virtual environment. To learn about what is conda: https://docs.conda.io/projects/miniconda/en/latest/miniconda-install.html Also read the official ''conda'' documentations.

Check in your terminal if conda has been installed, if not, you can either ask me to install it, or install it by yourself following the guides below.

Installation guides:

• We prefer to insall miniconda because it is ~480M, while anaconda is ~4.4GB. Check the difference between ''anaconda'' and ''miniconda''.
• Install to the default place, whith is /home/<username>/miniconda/.
• Following the official standard installation guide
• Or use this Quick Command Line Install.

Environment management:

• Create new environment from yml link.
• Update from yml link
• Delete env link
• Reverse to a previous version link
• Export yml for cross-platform link
  • conda env export –from-history –no-builds > environment-filename.yml

Issue: Not setting environment parameters properly, so cannot find CUDA installation or some others

A workaround that uses just one line command and needs to manually configured for each environment:

# with the desired conda environment activated
conda env config vars set LD_LIBRARY_PATH=$CONDA_PREFIX/lib

(credits: a GitHub issue discussion) Conda official documents about setting environment variables.

This has been a known issue of PyCharm for quite a long time. (For example: PyCharm doesn't set LD_LIBRARY_PATH when running scripts)

This allows the terminal to find Tensorflow path correctly in the conda environment, but not PyCharm yet. See Configure PyCharm to run TensorFlow for additional settings.

To use TensorFlow properly, LD_LIBRARY_PATH needs to be exposed to PyCharm properly. Within the correct conda env activated, find the correct path using:

echo $CONDA_PREFIX/lib

Then, add the output result as an environment variable LD_LIBRARY_PATH in PyCharm.

• Edit Run/Debug Configurations, add an environment variable with the name LD_LIBRARY_PATH and the value of the output above.
  • With this, when the code is executed in Run panel, it detects the variable.
• File | Settings | Build, Execution, Deployment | Console | Python Console, add the same environment variable above.
  • With this, when the code is executed in Python Console panel, it detects the variable.