Create DFT Environments

Create DFT Environments#

QUANTUM ESPRESSO 7.0+ Environment Setup#

on PACE

by Gabriel S. Gusmão (Mar, 2022)

gusmaogabriels@gmail.com/gusmaogabriels@gatech.edu

Knowledge Requirements#

  • Fortran Compilers / MPI / Git / PBS (SLURM/TORQ)

To setup you Quantum Espresso (QE) environment you will need the (i) standard Atomic Simulation Environment - ASE package, (ii) ase-espresso with modified classes for QE that enables interactions with live QE sessions through i-PI socket, and, naturally, (iii) Quantum Espresso itself.

Walk-through#

Assume `ROOT_DIR` is some shared path to which all users have access. The following file structure is arbitrary, but the environment file should make reference to the correct ones.

  1. Clone QUANTUM ESPRESSO into $ROOT_DIR/builds/espresso/7.0/, e.g. git clone git@github.com:QEF/q-e.git $ROOT_DIR/builds/espresso/7.0

  2. Compile QUANTUM ESPRESSO with the appropriate flags and loaded modules running, e.g., bash ggusmao_build.ssh from $ROOT_DIR/builds/espresso/7.0.

  • ggusmao_build.sh (Create this file in QE’s root folder)

    module purge
module load intel/19.0.5 mvapich2/2.3.2 gcc-compatibility/8.3.0 #netlib-scalapack/2.1.0-mva2 #anaconda3

export CC=icc
export CXX=icpc
export FC=mpif90
export F77=ifort
export F90=ifort
export MPIVERSION=mvapich2/2.3.2
export COMPILERVERSION=intel/19.0.5

make veryclean;

./configure --prefix=$(pwd) --with-scalapack=intel BEEF_LIBS="-L$LIBBEEF/src -lbeef" --enable-parallel=yes SCALAPACK_LIBS="$MKLROOT/lib/intel64/libmkl_scalapack_lp64.a -lmkl_blacs_intelmpi_lp64" BLAS_LIBS="-lmkl_gf_lp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_lp64 -lpthread" LAPACK_LIBS="-lmkl_lapack95_lp64 -lmkl_scalapack_lp64" CC=icc FCL="mpif90 -mkl=sequential -lstdc++"  FFLAGS="-assume byterecl -w" FLAGS="-O2" CFLAGS="-O3" FREE="-free -names lowercase" OBJECTS="fftmpiw.o fftmpi_map.o fft3dlib.o fftw3d.o"; make all -j12; make install

  1. Clone ASE into $ROOT_DIR/gits/dev/ase-latest/, e.g., git clone git@gitlab.com:ase/ase.git $ROOT_DIR/gits/dev/ase-latest/.

  2. Clone ase-espresso into $ROOT_DIR/gits/dev/ggusmao-espresso, e.g., git clone git@github.com:gusmaogabriels/ase-espresso.git ROOT_DIR/gits/dev/ggusmao-espresso.

  3. Create a module to handle pseudopotentials in $ROOT_DIR/data/pseudos_module/pseudos.env

  • getpseudo.py (This script scrapes for pseudopotential files within the current directory)

    import re
import os
import sys

def getpseudo(pseudo):
    pseudos_dir = '/'.join((os.environ['MEDFORD_SHARE'],'data/pseudos_module'))
    curdir = os.getcwd()
    os.chdir(pseudos_dir)
    pseudo_path = pseudo.replace('_','/').replace('#','_')
    paths = []
    for (dirpath, dirname, filename) in os.walk('.'):
        if any(list(filter(lambda x : x.lower().endswith('.upf') or x.lower().endswith('.psp8') or x.lower().endswith('.pot'), os.listdir(dirpath)))) and dirpath!='esp_pseudos':
            if pseudo_path == dirpath[2:]:
                pseudo_path = '/'.join((pseudos_dir,pseudo_path))
                os.chdir(curdir)
                return pseudo_path
            else:
                paths += [dirpath[2:].replace('_','#').replace('/','_')]

    os.chdir(curdir)
    if pseudo != '':
        raise(Exception('PSEUDO_NOT_FOUND\n\n\t{}\n\n'.format(pseudo)+
                     'Failed!\n\nAvailable pseudo potentials:\n\n{}\n'.format('\n'.join(paths))))
    else:
        return paths

  • setpseudo.py (This script defintes the pseudopotential path in the current environment)

    import re
import os
import sys
pseudo = sys.argv[1]

pseudos_dir = '/'.join((os.environ['ROOT_DIR'],'data/pseudos_module'))
curdir = os.getcwd()
os.chdir(pseudos_dir)
pseudo_path = pseudo.replace('_','/').replace('#','_')
paths = []
for (dirpath, dirname, filename) in os.walk('.'):
    if any(list(filter(lambda x : x.lower().endswith('.upf') or x.lower().endswith('.psp8') or x.lower().endswith('.pot'), os.listdir(dirpath)))) and dirpath!='esp_pseudos':
        if pseudo_path == dirpath[2:]:
            pseudo_path = '/'.join((pseudos_dir,pseudo_path))
            sys.stdout.write(pseudo_path)
            os.chdir(curdir)
            sys.exit('PSP directory successfully set: {}'.format(pseudo_path))
        else:
            paths += [dirpath[2:].replace('_','#').replace('/','_')]

os.chdir(curdir)
if pseudo != '':
    raise(Exception('PSEUDO_NOT_FOUND\n\n\t{}\n\n'.format(pseudo)+
                 'Failed!\n\nAvailable pseudo potentials:\n\n{}\n'.format('\n'.join(paths))))
else:
    sys.exit('\nAvailable pseudo potentials:\n\n{}\n'.format('\n'.join(paths)))

  • pseudos.env (This script loads the python scripts as bash functions)

    function setpseudos()
{
    export ESP_PSP_PATH=$(python <current_path>/setpseudo.py "$1")
    export SPARC_PSP_PATH=$(echo $ESP_PSP_PATH)
}

Running setpseudos after sourcing pseudos.env gives a list of available pseudo-potentials. You can download pseudopotentials from standard libraries, e.g., pseudojodo, SSSP, etc.

  1. Create an environment file to load all paths into the current session

  • quantum_espresso-7.0 (This is your environment file that need be sourced before you run any code)

    #Load PACE-wide modules

module purge
module load intel/19.0.5 mvapich2/2.3.2 gcc-compatibility/8.3.0 anaconda3 

pip install py-hostlist --user 

cd ~
export HOME=$(pwd)
cd -
export ROOT_DIR=<ROOT_DIRECTORY_PATH>
source $ROOT_DIR/data/pseudos_module/pseudos.env
setpseudos espresso_esp#pseudos#old # <- you can redefine your PPS path accordingly
export SCRATCH=$HOME/scratch  

#get our group base environment

#Set up program-specific paths
export PYTHONPATH=$ROOT_DIR/gits/dev/ase-latest/:$PYTHONPATH
export PYTHONPATH=$ROOT_DIR/gits/dev/ggusmao-espresso:$PYTHONPATH
export PYTHONPATH=$ROOT_DIR/gits/dev/spglib:$PYTHONPATH # OPTIONAL
export PYTHONPATH=$ROOT_DIR/data/pseudos_module:$PYTHONPATH
export DATA=$HOME/data

#group specific commands
export PATH=$ROOT_DIR/INSTALL/bin:$PATH
#get quantum espresso
export PATH=$ROOT_DIR/builds/espresso/7.0/bin:$PATH
#get ase_latest
export PATH=$ROOT_DIR/gits/dev/ase-latest/bin:$PATH

Example Calculation#

on PACE

  • run.sh (bash script to run simulation on PACE HIVE)

---
```
#!/bin/bash
#PBS -l nodes=1:ppn=8
#PBS -l pmem=3GB
#PBS -l walltime=12:00:00
#PBS -A GT-amedford6
#PBS -q hive
#PBS -N optimizer
#PBS -o stdout
#PBS -e stderr

cd $PBS_O_WORKDIR
source ~/.bashrc
module load anaconda3
source /storage/coda1/p-amedford6/0/shared/rich_project_chbe-medford/medford-share/envs/espresso-6.7MaX-beef-ipi
python calc.py
```
---

---
```
from ase import Atom, Atoms
from ase.optimize import BFGSLineSearch as BFGS
from ase.build import molecule
from espresso import iEspresso as Espresso
import numpy as np
# from ase.io import write

import time

t0 = time.time()
images = []
image = molecule('H2O',vacuum=10.)
image.set_cell([10, 10, 10])
image.set_pbc([1,1,1])
image.center()

#Use QE as a single-point calculator (just SCF)
calc = Espresso(atoms=image,
                pw=500.0,
                xc='PBE')

# retrieve Energy, force, stress
image.rattle(0.0001)
print('1st calculation with PBE')
print('Energy = {}'.format(image.get_potential_energy()))
print('Force = {}'.format(image.get_forces()))
print('Stress = {}'.format(image.get_stress()))

t1 = time.time()

print('Time = {} sec'.format(t1-t0))

print('-'*20)

# Run optimization
dyn = BFGS(image,logfile='opt_pbe.log', trajectory='h2o_pbe_optimization.traj')
dyn.run(fmax=0.005)

image.calc.close()

# Attach BEEF-vdw calculator
t2 = time.time()
calc = Espresso(atoms=image,
                pw=350.0,
                xc='BEEF-vdw')

# retrieve Energy, force, stress after optimization
print('2nd calculation')
print('Energy = {}'.format(image.get_potential_energy()))
print('Force = {}'.format(image.get_forces()))
print('Stress = {}'.format(image.get_stress()))
E_ensemble = image.get_ensemble_energies()
print('Ensemble energies = {} +/- {}'.format(E_ensemble.mean(), E_ensemble.std()))
print('Time = {} sec'.format(time.time()-t2))

# Optimization
dyn = BFGS(image,logfile='opt_beef.log',trajectory='h2o_beef_optimization.traj')
dyn.run(fmax=0.005)
image.calc.close()
# write('water.extxyz', images)
print('Totol time = {} sec'.format(time.time()-t0))
```
---

LRT Compile#

  1. Begin by downloading *.zip files for QE and libxc. Using QE_7.0 and libxc_6.0.0

  • wget https://gitlab.com/libxc/libxc/-/archive/6.0.0/libxc-6.0.0.zip

  • wget https://gitlab.com/QEF/q-e/-/archive/qe-7.0/q-e-qe-7.0.zip

  • It appears that libbeef is automatically included in QE 6.6 or later.

  1. Compile libxc

  • Begin an interactive job

  • LRT_build.sh (Create this file in libxc’s root folder)

    module purge
module load intel/20.0.4 mvapich2/2.3.6

autoreconf -i

./configure --prefix=PATH/TO/LIBXC
make -<N processors>
make check
make install

  1. Compile QE

  • Either begin or continue interactive job

  • LRT_build.sh (Create this file in QE’s root folder)

    module purge
module load intel/20.0.4 mvapich2/2.3.6

./configure --enable-shared=yes --with-scalapack=intel --with-libxc --with-libxc-prefix=/PATH/TO/LIBXC
make all -j<N processors>

End