oheil / affycelfiles.jl

read Affymetrix .CEL files

AffyCelFiles

This is a low level IO package for Affymetrix .CEL files

References

https://media.affymetrix.com/support/developer/powertools/changelog/file-formats.html https://media.affymetrix.com/support/developer/powertools/changelog/gcos-agcc/cel.html https://media.affymetrix.com/support/developer/powertools/changelog/gcos-agcc/generic.html https://media.affymetrix.com/support/developer/powertools/changelog/gcos-agcc/cdf.html https://media.affymetrix.com/support/developer/powertools/changelog/file-format-pgf.html https://media.affymetrix.com/support/developer/powertools/changelog/file-format-clf.html https://media.affymetrix.com/support/developer/powertools/changelog/file-format-mps.html

Currently supported microarrays

This package is in an early stage and the following microarrays .CEL files are tested:

• Clariom_S_Human
• Mouse430_2
• HG-U133_Plus_2
• HuGene-2_0-st

(more to come)

The expression/intensity data was cross checked against R/bioconductor affy and oligo libraries.

Dependencies

Julia versions

• Julia 1.0 or above

Third party packages

• none

Standard Library packages

• CRC32c

Usage

In general Affymetrix .CEL can always be read in. To read Affymetrix .CEL files with meaningful information you need one of the following additional file/files:

• .cdf
• .pgf + .clf (+ optional .mps)

Those files provide the mapping from a probe location on the chip to a probe_id and the mapping from probeset_ids to probe_ids. For a specific chip type you can get those files from the ThermoFisher support page, e.g. for the Clariom S human see https://www.thermofisher.com/order/catalog/product/902927?SID=srch-srp-902927 for download of archive TFS-Assets_LSG_Support-Files_Clariom_S_Human_Analysis-r1.zip (you need to have a login) which contains the .pgf,.clf and .mps file for this chip.

For biological meaningful analysis you also need annotation data, which maps probeset_ids to, for example, gene names. This is not part of this package, but affymetrix annotation data is typically just a file of annotations in a table/csv format which can easily be read in (e.g. with CSV.jl) and mapped using probeset_ids or transcript_ids/meta_probeset_ids (in case of .mps used).

In this early stage this package doesn't provide a very convenient API, but this may change in future.

Usage examples

using Pkg
Pkg.add("AffyCelFiles");
#or from github main:
#Pkg.add(url="https://github.com/oheil/AffyCelFiles.jl",rev="main")

using AffyCelFiles

Reading .CEL files with corresponding .cdf file:

cel_file = "example_(Mouse430_2).CEL"
cdf_file = raw"d:\Temp\mouse430_2_libraryfile\CD_Mouse430_2\Full\Mouse430_2\LibFiles\Mouse430_2.cdf"

cel_data = AffyCelFiles.cel_read(cel_file);
cdf_data = AffyCelFiles.cdf_read(cdf_file);

intensities = AffyCelFiles.intensities(cel_data,cdf_data);

#Dict mapping probeset_id to lists of expression values (pm=perfect match, mm=mismatch).
#Depending on the chiptype mm can be empty
intensities.pm
intensities.mm

Result:

julia> intensities.pm
Dict{String, Vector{Float32}} with 45101 entries:
  "1453553_at"   => [93.0, 316.0, 74.0, 214.0, 94.0, 112.0, 52.0, 319.0, 66.0, 74.0, 88.0]
  "1455273_at"   => [1054.0, 57.0, 102.0, 263.0, 180.0, 447.0, 498.0, 683.0, 1233.0, 196.0, 999.0]
  "1425429_s_at" => [59.0, 162.0, 79.0, 286.0, 114.0, 384.0, 305.0, 111.0, 106.0, 138.0, 572.0]
  "1445844_at"   => [129.0, 749.0, 155.0, 156.0, 51.0, 144.0, 53.0, 123.0, 136.0, 102.0, 63.0]
  "1443594_at"   => [82.0, 150.0, 306.0, 447.0, 392.0, 101.0, 79.0, 143.0, 108.0, 531.0, 70.0]
  "1449219_at"   => [123.0, 104.0, 118.0, 74.0, 76.0, 136.0, 116.0, 165.0, 175.0, 116.0, 88.0]
  "1447933_at"   => [68.0, 120.0, 164.0, 42.0, 57.0, 50.0, 228.0, 272.0, 92.0, 68.0, 65.0]
...
julia> intensities.mm
Dict{String, Vector{Float32}} with 45101 entries:
  "1453553_at"   => [124.0, 75.0, 56.0, 317.0, 69.0, 165.0, 43.0, 148.0, 100.0, 65.0, 143.0]
  "1455273_at"   => [171.0, 51.0, 284.0, 93.0, 64.0, 174.0, 284.0, 233.0, 169.0, 48.0, 119.0]
  "1425429_s_at" => [51.0, 200.0, 80.0, 135.0, 94.0, 254.0, 255.0, 98.0, 93.0, 61.0, 161.0]
  "1445844_at"   => [79.0, 255.0, 105.0, 97.0, 65.0, 90.0, 44.0, 152.0, 187.0, 56.0, 40.0]
  "1443594_at"   => [57.0, 171.0, 181.0, 522.0, 244.0, 70.0, 70.0, 87.0, 80.0, 1093.0, 53.0]
...

Reading .CEL files with corresponding .pgf and .clf files (you need both):

cel_file = "example_(Clariom_S_Human).CEL"
pgf_file = raw"d:\Temp\clariomShuman\TFS-Assets_LSG_Support-Files_Clariom_S_Human_Analysis-r1\Clariom_S_Human.r1.pgf"
clf_file = raw"d:\Temp\clariomShuman\TFS-Assets_LSG_Support-Files_Clariom_S_Human_Analysis-r1\Clariom_S_Human.r1.clf"

cel_data = AffyCelFiles.cel_read(cel_file);
pgf_data = AffyCelFiles.pgf_read(pgf_file);
clf_data = AffyCelFiles.clf_read(clf_file);

intensities = AffyCelFiles.intensities(cel_data, pgf_data, clf_data);

#Dict mapping probeset_id to lists of expression values (pm=perfect match, mm=mismatch).
#Depending on the chiptype mm can be empty
intensities.pm
intensities.mm

Result:

julia> intensities.pm
Dict{String, Vector{Float32}} with 27189 entries:
  "23050629" => [61.0, 43.0, 54.0, 60.0]
  "23054888" => [321.0, 184.0, 169.0, 297.0, 162.0, 217.0, 194.0, 647.0, 124.0, 192.0]
  "23060812" => [56.0, 46.0, 145.0, 58.0, 92.0, 80.0, 204.0, 152.0, 83.0, 118.0]
  "23056335" => [160.0, 227.0, 951.0, 264.0, 286.0, 52.0, 43.0, 53.0, 48.0, 40.0]
  "23059041" => [1043.0, 867.0, 321.0, 739.0, 826.0, 462.0, 330.0, 1190.0, 1485.0, 1537.0]
...
julia> intensities.mm
Dict{String, Vector{Float32}} with 27189 entries:
  "23050629" => []
  "23054888" => []
  "23060812" => []
...

Providing a .mps file changes the central ids from probeset_ids to meta_probeset_ids/transcript_ids. Reading .CEL files with corresponding .pgf, .clf files (you need both) and optional a .mps file:

cel_file = "example_(Clariom_S_Human).CEL"
pgf_file = raw"d:\Temp\clariomShuman\TFS-Assets_LSG_Support-Files_Clariom_S_Human_Analysis-r1\Clariom_S_Human.r1.pgf"
clf_file = raw"d:\Temp\clariomShuman\TFS-Assets_LSG_Support-Files_Clariom_S_Human_Analysis-r1\Clariom_S_Human.r1.clf"
mps_file = raw"d:\Temp\clariomShuman\TFS-Assets_LSG_Support-Files_Clariom_S_Human_Analysis-r1\Clariom_S_Human.r1.mps"

cel_data = AffyCelFiles.cel_read(cel_file);
pgf_data = AffyCelFiles.pgf_read(pgf_file);
clf_data = AffyCelFiles.clf_read(clf_file);
mps_data = AffyCelFiles.mps_read(mps_file);

intensities = AffyCelFiles.intensities(cel_data, pgf_data, clf_data, mps_data);

#Dict mapping meta_probeset_id/transcript_id to lists of expression values (pm=perfect match, mm=mismatch).
#Depending on the chiptype mm can be empty
intensities.pm
intensities.mm

Result:

julia> intensities.pm
Dict{String, Vector{Float32}} with 24351 entries:
  "TC0600009248.hg.1" => [42.0, 35.0, 195.0, 43.0, 140.0, 207.0, 54.0, 111.0, 95.0, 70.0]
  "TC0800011018.hg.1" => [47.0, 46.0, 41.0, 51.0, 74.0, 43.0, 62.0, 45.0, 42.0, 34.0]
  "TC0500012822.hg.1" => [299.0, 189.0, 425.0, 337.0, 1465.0, 1790.0, 988.0, 784.0, 471.0, 1094.0]
  "23050629"          => [61.0, 43.0, 54.0, 60.0]
  "TC1200012657.hg.1" => [2004.0, 1886.0, 450.0, 374.0, 175.0, 893.0, 332.0, 1710.0, 1930.0, 1362.0]
  "TC0700012299.hg.1" => [129.0, 77.0, 53.0, 76.0, 96.0, 395.0, 73.0, 130.0, 95.0, 82.0]
...

affycelfiles.jl's Issues

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!