Commands to Access Local Data

Contents

• Introduction
• Copying data from DFS
• Accessing data from DFS
• Accessing_data inside ROOT
  • Open a single file in ROOT
  • Open a multiple file in ROOT
• Accessing_data using gfal2 tools
  • Creating a directory in EOS
  • Delete an empty directory in EOS
  • Copy data from local to EOS
  • Delete data from EOS
  • List EOS data directory
  • Renaming a folder in EOS

Introduction

Starting in 2023 the CMS Tier-2 at Purdue uses EOS as a local distributed filesystem for storing users' and experiment's data.

The local Distributed Filesystems (DFS) is intended and optimized for storing large (ROOT) datafiles, and not for small text files (e.g. source code). XRootD protocol should be the main access method, using the gfal-* and xrd* commands described below.

Accessing data via xrootd

Data in DFS can be read from 'eos.cms.rcac.purdue.edu' with a valid grid certificate.

xrdcp root://eos.cms.rcac.purdue.edu//store/path/to/file.root  /path/to/my.root 

Accessing data inside ROOT

Use the following methods to load files in ROOT.

Open a single file in ROOT

root [0] TFile* f = TFile::Open("root://eos.cms.rcac.purdue.edu//store/group/ewk/DY/May10ReReco/ntuple_skim_9_6_ZgF.root");

Open multiple files in ROOT

Create text file in the directory you are going to start ROOT, call it "xrootdfiles.txt". Populate xrootdfiles.txt with the files you would like to open.

root://eos.cms.rcac.purdue.edu//store/path/to/file1.root
root://eos.cms.rcac.purdue.edu//store/path/to/file2.root
.
.
.
root://eos.cms.rcac.purdue.edu//store/path/to/fileX.root

Add the files to a TFileCollection object in ROOT:

root [0] TFileCollection* c1 = new TFileCollection("data","data");
root [1] c1->AddFromFile("xrootdfiles.txt");
root [2] c1->Print("L");
TFileCollection data - data contains: 3 files with a size of 0 bytes, 
0.0 % staged - default tree name: '(null)'

Accessing data using gfal tools

Data at the Purdue Tier-2 can be accessed from 'eos.cms.rcac.purdue.edu' with a valid grid certificate.

# save the original path into a new environment variable
export LD_LIBRARY_PATH_DEFAULT=$LD_LIBRARY_PATH

# run your work that requires cmsenv
# ...

# reset the LD_LIBRARY_PATH environment variable
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH_DEFAULT

Creating a directory in EOS

In the following commands <username> stands for your CERN lxplus ID.

gfal-mkdir davs://eos.cms.rcac.purdue.edu:9000/store/user/<username>/<directoryname>

Delete a directory and all its contents in EOS (be careful!)

gfal-rm -r davs://eos.cms.rcac.purdue.edu:9000/store/user/<username>/<directoryname>

Copy data from local to EOS

gfal-copy file:////tmp/test.root davs://eos.cms.rcac.purdue.edu:9000/store/user/<username>/test.root

Delete data from EOS

gfal-rm davs://eos.cms.rcac.purdue.edu:9000/store/user/<username>/test.root

List EOS data directory

gfal-ls davs://eos.cms.rcac.purdue.edu:9000/store/user/<username>

Renaming a folder in EOS

gfal-rename davs://eos.cms.rcac.purdue.edu:9000/store/user/<username>/folder1 davs://eos.cms.rcac.purdue.edu:9000/store/user/<username>/folder2