PIP-ATAC-Pipeline

Snakemake pipeline for processing PIP-ATAC data. It will submit mulitple jobs for each sample in parallel.

Requirements

• Conda
• Snakemake
• snakemake-executor-plugin-slurm # for running on slurm

Structure

• workflow/ – Snakemake rules
• config/ – configuration files.
• envs/ – conda environment YAMLs
• data/ – input data (fastq.gz files. need R1 R2 Index1 Index2)
• results/ – output files
• logs/ – log files

Demultiplex

If starting with BCL files, first convert them to FASTQ.gz format using bcl-convert. Index files will need to be generated, which can be done using a sample sheet. An example sample sheet is shown below. Please adjust the read lengths accordingly.

[Header],
Experiment Name,260216_pbmc_brain_ATAC
Date,260216
Workflow,GenerateFASTQ

[Reads]
45
75
10
75

[Settings]
CreateFastqForIndexReads,1
NoLaneSplitting,TRUE
OverrideCycles,Y45;U75;I10;Y75
TrimUMI,0

[Data]
Sample_ID,index2
ZL204_Brain_1_ATAC_2,TCGTGGAGCG
ZL205_Brain_2_ATAC_2,CTACAAGATA
ZL206_PBMC_1_ATAC,GGCCTGTCCT
ZL207_PBMC_2_ATAC,CTGTGTTAGG

Usage

Clone the repository into a pipeline folder. On Stanford Sherlock, it is recommended to place it on SCRATCH because of faster I/O. Put all fastq.gz files in the data folder. In the config folder, edit the paths for the PIPseeker software and Bowtie2 references in config.yaml. Update the samples.tsv file accordingly. Add the sample names and the reference genome names. In the example below, the sample name is the part before "_S", which is ZL206_PBMC_1_ATAC.

ZL206_PBMC_1_ATAC_S3_I1_001.fastq.gz
ZL206_PBMC_1_ATAC_S3_I2_001.fastq.gz
ZL206_PBMC_1_ATAC_S3_R1_001.fastq.gz
ZL206_PBMC_1_ATAC_S3_R2_001.fastq.gz

The amount of resources allocated to each step, as well as the partition name, can be adjusted by editing config.yaml under profiles/slurm

It's recoomend to have a dry run first and check the output first using the below command.

snakemake --profile profiles/slurm -n

Run the below snakemake command to launch the jobs.

snakemake --profile profiles/slurm --printshellcmds

Because the pipeline can take a long time to complete, it is recommended to submit a bash script as a job to launch Snakemake. An example is provided below.

#!/bin/bash
#SBATCH --job-name=snakemake_main
#SBATCH --partition=wjg
#SBATCH --time=40:00:00
#SBATCH --mem=4G
#SBATCH --cpus-per-task=2
#SBATCH --output=logs/snakemake_main_%j.out
#SBATCH --error=logs/snakemake_main_%j.err
#SBATCH --mail-type=ALL
#SBATCH --mail-user=xxxx@stanford.edu


# Activate conda
source xxx/miniconda3/etc/profile.d/conda.sh
conda activate snakemake

# Navigate to project
cd xxx/pipeline

# Run Snakemake
snakemake \
--profile profiles/slurm \
--printshellcmds \
2>&1 | tee logs/snakemake_$(date +%Y%m%d_%H%M%S).log

In the end, it generates a fragment file which used a +4/-4bp offset approach to create a consensus insertion site for downstream analysis.