Index of /examples/bioinformatics/advanced/array_job_in_bio

Name	Last modified	Size

Parent Directory		-
out/	2023-05-08 14:39	-
in/	2023-05-08 14:39	-
qsub/	2023-05-08 14:38	-
qlog/	2023-05-08 14:37	-

RCS ARRAY JOB Example For Bioinformatics

Directory Structure

in - a dedicated folder to put all sample fastq files of the workflow.
in/*.fq - the input sequence fastq files to be processed.
out/ - a dedicated folder to store the output from the workflow.
qsub dedicated folder to store all qsub scripts.
qsub/bio_array_job.qsub - Batch job script example
qlog dedicated folder to store all qsub output logs.

Notes

To view all available versions of samtools on SCC, execute:
[scc1 ] module avail samtools

To run samtools view to convert sam to bam:
[scc1 ] module load samtools/1.10 # please specify version
[scc1 ] samtools view -bSu sam_in/example_aln.sam -o sam_out/example_aln.bam

The above command will run silently (no output on the screen). And it will generate a bam file in sam_out/example_aln.bam.

sam_out/example_aln.bam

To then use 'sort' function of the samtools to sort the bam:
[scc1 ] samtools sort sam_out/example_aln.bam -o sam_out/example_aln_sorted.bam

The above command will execute silently and produce a sorted bam file in sam_out/example_aln_sorted.bam.

sam_out/example_aln_sorted.bam

Please note, samtools can support multithreads automactically. To make use of the function, use '--threads' or '-@' option to specify the cores (threads) to use:
[scc1 ] module load samtools/1.10 # please specify version

[scc1 ] samtools view --threads 2 -bSu sam_in/example_aln.sam | samtools sort  --threads 2 -o sam_out/example_aln_sorted_2.bam

The above command is a combination of the two samtools functions - view and sort, using '|' (pipe symbol) to feed the second command with the first command's output. This is a common technique used in Linux to simplify I/O and improve efficiency. And it specifies 2 threads for each command (--threads 2).

sam_out/example_aln_sorted_2.bam

You can write a qsub to combine the above steps for a batch submission:
As prepared in qsub/samtools_job.qsub
[scc1 ] cd qsub # get into the qsub script location
[scc1 ] qsub samtoools_job.qsub

After the above job's completion, there will be two files in qlog/ looks like:

qlog/samtools_example.oxxxxxxx
qlog/samtools_example.poxxxxxxx # 'xxxxxxx' stands for job number

And there will be a sorted bam file newly generated in sam_out/:

sam_out/samtools_example.sorted.bam

Samtools links

Contact Information

Research Computing Services: help@scc.bu.edu

Note: RCS example programs are provided "as is" without any warranty of any kind. The user assumes the intire risk of quality, performance, and repair of any defect. You are welcome to copy and modify any of the given examples for your own use.