Non-concussive head impacts sustained during American football correlate with changes in gut microbiome diversity and composition.
The 12 matches
- [1] § Materials and methods › Fecal sample collection lifestyle questionnaires ↔ TBI_CLR.Rmd, lines 30–80 · score 0.82 · miceRanger, Bristol Stool, orthopedic injury, illness, nicotine, vomiting
- [2] § Materials and methods › Fecal sample collection lifestyle questionnaires ↔ TBI_Microbiome_R.Rmd, lines 31–83 · score 0.82 · miceRanger, Bristol Stool, orthopedic injury, illness, nicotine, vomiting
- [3] § Results › Bray-Curtis dissimilarity increases three days following substantial head impact exposure ↔ TBI Microbiome.ipynb, lines 1033–1165 · score 0.73 · 72–96 hours, 0–24 hours, Bray Curtis Dissimilarity, 48–72 hours, head impact, Nemenyi
- [4] § Results › Bray-Curtis dissimilarity increases three days following substantial head impact exposure ↔ TBI_Microbiome.ipynb, lines 978–1109 · score 0.73 · 72–96 hours, 0–24 hours, Bray Curtis Dissimilarity, 48–72 hours, head impact, Nemenyi
- [5] § Results › Bray-Curtis dissimilarity increases three days following substantial head impact exposure ↔ TBI Microbiome.ipynb, lines 1033–1165 · score 0.68 · 72–96 hours, 0–24 hour, Bray Curtis Dissimilarity, 48–72, head impact, Nemenyi
- [6] § Results › Bray-Curtis dissimilarity increases three days following substantial head impact exposure ↔ TBI_Microbiome.ipynb, lines 978–1109 · score 0.68 · 72–96 hours, 0–24 hour, Bray Curtis Dissimilarity, 48–72, head impact, Nemenyi
- [7] § Materials and methods › Data analysis – Data slicing and repeated-measures analysis ↔ TBI Microbiome.ipynb, lines 920–1029 · score 0.57 · post hoc pairwise, Rank, ANOVA, Nemenyi, Friedman, microbial
- [8] § Materials and methods › Data analysis – Data slicing and repeated-measures analysis ↔ TBI_Microbiome.ipynb, lines 865–974 · score 0.57 · post hoc pairwise, Rank, ANOVA, Nemenyi, Friedman, microbial
- [9] § Materials and methods › Data processing – Taxonomic data ↔ TBI Microbiome.ipynb, lines 273–291 · score 0.55 · Silva V4, classifier provided, taxonomic, sequence, microbial
- [10] § Materials and methods › Data processing – Taxonomic data ↔ TBI_Microbiome.ipynb, lines 280–298 · score 0.55 · Silva V4, classifier provided, taxonomic, sequence, microbial
- [11] § Results › Gut microbiome composition changes 48–72 hours post head impact ↔ TBI_CLR.Rmd, lines 251–321 · score 0.55 · CLR, circumstances, Orthopedic, caffeine, Coriobacteriales, sleep
- [12] § Results › Gut microbiome composition changes 48–72 hours post head impact ↔ TBI_Microbiome_R.Rmd, lines 312–385 · score 0.51 · circumstances, Orthopedic, caffeine, Coriobacteriales, sleep, Verrucomicrobiales
Paper
Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC
The paper is loaded when this pane is shown.
The authors' code
Jupyter notebook · 1,633 lines · 70 KB · no license · 4 matches
- # %% [markdown]
- # # General Notes
- # %% [markdown]
- # This Jupyter Notebook mostly uses command line commands instead of actual Python code to utilize QIIME 2. In the first line of code cells, you can often see `%%bash`, which is used to denote that the chunk contains commands rather than Python code.
- #
- # QIIME 2 has its own file formats: `.qza` and `.qzv`. You can consider `.qza` as files that contain data and `.qzv` files as visualizations of results. To view results stored in `.qzv` files, open [QIIME View](https://view.qiime2.org) (https://view.qiime2.org) and drag your `.qzv` file into the browser window. The file will load and display results.
- #
- # Make sure you are running this notebook with your QIIME 2 virtual environment. If you have followed QIIME 2's installation guide and installed successfully, you should have a dedicated Anaconda/Miniconda virtual environment for QIIME 2. If you are using this notebook with VSCode, select the environment that corresponds to QIIME 2 on the upper right side.
- #
- # If you have any questions about QIIME 2, they have detailed [documentation](https://docs.qiime2.org) online. Specifically for this notebook (time series analysis), many of the commands use the `longitudinal` module. You can check the tutorial on [longitudinal analysis](https://docs.qiime2.org/2022.11/tutorials/longitudinal/) specifically if you want more details on the commands.
- # %% [markdown]
- # # Setup
- # %% [markdown]
- # ### Import Libraries
- # %%
- %matplotlib inline
- import pandas as pd
- import os
- import numpy as np
- #import xlrd
- import matplotlib.pyplot as plt
- from sklearn import metrics
- from sklearn.linear_model import LinearRegression
- #import statsmodels.api as sm
- from sklearn.metrics import r2_score
- import seaborn as sns
- #import statsmodels.formula.api as smf
- #import patsy
- from typing import Union
- from scipy import stats
- from itertools import combinations
- from scipy.spatial.distance import braycurtis, pdist, squareform
- from scipy.stats import chi2, rankdata
- #from statsmodels.stats.anova import AnovaRM
- #from statsmodels.stats.libqsturng import psturng
- #from skbio import DistanceMatrix
- #from skbio.stats.ordination import pcoa
- #import scikit_posthocs
- import math
- import datetime
- import warnings
- warnings.filterwarnings('ignore')
- # %% [markdown]
- # ### Change working directory to your folder
- # %%
- #workdir='/Users/Aziz/Documents/ApMa Thesis/2023_03_14_012623KBillcus515F_Raw_Data_UDI/data_files' # Set this to your working directory
- workdir = '/Users/aziz/Documents/Ay Lab/2023_03_14_012623KBillcus515F_Raw_Data_UDI/data_files'
- %cd $workdir
- # %% [markdown]
- # # Load Files
- # %% [markdown]
- # ### Zip fastq Files
- # %% [markdown]
- # If you have files in .fastq format, run this chunk to zip it to .fastq.gz format so that importing is more efficient.
- #
- # If you already have files in .fastq.gz format, skip this chunk.
- # %%
- %%bash
- gzip FastQ/*.fastq
- # %% [markdown]
- # ### Construct Manifest File and convert Metadata
- # %% [markdown]
- # You need to manually create a manifest file, which specifies the file paths of .fastq.gz files for each sample.
- #
- # If you have paired end data, create this file in excel with three columns: "sampleid", "forward-absolute-filepath", "reverse-absolute-filepath". If you have single end data, you only need the first two columns.
- #
- # List the sample names in the "sampleid" column, and put the absolute file paths to the sample's forward and reverse reads in the next two columns. Then save the excel file as manifest.csv (make sure it is .csv), and run this chunk.
- #
- # For a reference of what a manifest file looks like, check QIIME 2 [documentation](https://docs.qiime2.org/2022.2/tutorials/importing/#fastq-manifest-formats).
- #
- # Below, there is some code to generate a manifest file using file names.
- # %%
- list_of_files = os.listdir(workdir)
- print(list_of_files)
- if '.DS_Store' in list_of_files:
- list_of_files.remove('.DS_Store')
- if "manifest.tsv" in list_of_files:
- os.remove("manifest.tsv")
- #print(list_of_files)
- separator = "_S1_"
- sample_ids = [filename.split(separator, 1)[0] for filename in list_of_files]
- sample_ids = [*set(sample_ids)]
- #print(sample_ids)
- forward_absolute_files = []
- reverse_absolute_files = []
- for id in sample_ids:
- #find the forward and reverse for this id
- forward_reverse = [filename for filename in list_of_files if id in filename]
- #print(forward_reverse)
- if "R1" in forward_reverse[0]:
- forward_absolute_files.append(os.path.join(workdir, forward_reverse[0]))
- reverse_absolute_files.append(os.path.join(workdir, forward_reverse[1]))
- else:
- forward_absolute_files.append(os.path.join(workdir, forward_reverse[1]))
- reverse_absolute_files.append(os.path.join(workdir, forward_reverse[0]))
- manifest_dict = {"sample-id": sample_ids, "forward-absolute-filepath": forward_absolute_files,
- "reverse-absolute-filepath": reverse_absolute_files}
- manifest = pd.DataFrame(data=manifest_dict)
- manifest.to_csv("manifest.tsv", sep = "\t", index = False)
- # %%
- manifest = pd.read_csv("manifest.csv")
- # manifest = manifest.iloc[:,:3] # This line of code keeps the first three columns, making sure there's no empty columns messing up the manifest. May not be necessary
- manifest.to_csv("manifest.tsv", sep="\t", index=False)
- # %% [markdown]
- # ### Import Sequences and View Quality Information
- # %% [markdown]
- # This chunk imports the sequences into QIIME.
- #
- # Make sure the `type` and `input-format` parameters are appropriate for your data. If you have single end data, change `type` to `SampleData[SequencesWithQuality]` and `input-format` to `SingleEndFastqManifestPhred33V2`.
- #
- # Make sure the offset for sequence quality score is 33. You should have this information from the people who conducted the sequencing when you get the sequences back. If the offset is 64 instead of 33, change the `33` to `64` in `input-format`.
- # %%
- %%bash
- qiime tools import --type 'SampleData[PairedEndSequencesWithQuality]' --input-path manifest.tsv --output-path demux.qza --input-format PairedEndFastqManifestPhred33V2
- # %% [markdown]
- # This next chunk visualizes the quality information. After you run this chunk, you should have a `demux.qzv` file in your folder. Open [QIIME View](https://view.qiime2.org) in your browser and drag `demux.qzv` in. Click on the `Interactive Quality Plot` tab at the upper left to view quality information.
- # %%
- %%bash
- qiime demux summarize \
- --i-data demux.qza \
- --o-visualization demux.qzv
- # %% [markdown]
- # # Sequence Trim and Denoise
- # %% [markdown]
- # There are two choices of denoising methods within QIIME: DADA2 and Deblur. There is no consensus on which is better, but you should have preferences depending on your data type.
- #
- # DADA2 works natively with paired end data, while Deblur does not. To use with Deblur, paired end sequences must be joined first with `qiime vsearch join-pairs` function (or some other joining action) to become joined sequences. However, parameters for the `vsearch join-pairs` operation are very tricky. They are not easily chosen and may lead to loss of read data.
- #
- # Typically, these two methods will give similar results for weighted analysis, but may have variations in unweighted analysis and alpha diversity metrics. DADA2 usually identifies noticeably more ASVs than Deblur in their output.
- #
- # DADA2 takes significantly longer and more memory to run than Deblur, so if you have huge single end read files, you may want to try Deblur first.
- #
- # Be aware that the outputted file names are different for these two methods in order to distinguish (e.g. dada2-table.qza and deblur-table.qza), so change file names accordingly for later analysis.
- # %% [markdown]
- # ### DADA2:
- # %% [markdown]
- # After viewing quality information in `demux.qzv`, you can determine the truncating spot on the left for forward (`p-trim-left-f`) and reverse (`p-trim-left-r`) reads. `p-trunc-len-f` and `p-trunc-len-r`are for truncating spots on the right for forward and reverse reads. `p-n-threads` determines how many cores of you CPU will be used. If you would like to use all cores, put `0` for this parameter.
- # %%
- %%bash
- qiime dada2 denoise-paired \
- --i-demultiplexed-seqs demux.qza \
- --p-trim-left-f 0 \
- --p-trim-left-r 0 \
- --p-trunc-len-f 220 \
- --p-trunc-len-r 180 \
- --p-n-threads 8 \
- --o-table dada2-table.qza \
- --o-representative-sequences dada2-rep-seqs.qza \
- --o-denoising-stats dada2-denoising-stats.qza
- # %%
- %%bash
- qiime feature-table summarize \
- --i-table dada2-table.qza \
- --o-visualization dada2-table.qzv \
- --m-sample-metadata-file metadata.tsv
- qiime feature-table tabulate-seqs \
- --i-data dada2-rep-seqs.qza \
- --o-visualization dada2-rep-seqs.qzv
- qiime metadata tabulate \
- --m-input-file dada2-denoising-stats.qza \
- --o-visualization dada2-denoising-stats.qzv
- # %% [markdown]
- # ### Deblur:
- # %% [markdown]
- # If you have paired end reads, join them first by running this chunk.
- #
- # For `qiime vsearch join-pairs` parameters, refer to [documentation](https://docs.qiime2.org/2022.2/plugins/available/vsearch/join-pairs/).
- # %%
- %%bash
- qiime vsearch join-pairs \
- --i-demultiplexed-seqs demux.qza \
- --p-truncqual 10 \
- --p-threads 0 \
- --o-joined-sequences deblur-demux.qza
- qiime demux summarize \
- --i-data deblur-demux.qza \
- --o-visualization deblur-demux.qzv
- # %% [markdown]
- # For `qiime deblur denoise-16S` parameters, check quality information in `deblur-demux.qzv`. `p-left-trim-len` is the truncating spot on the left, and `p-trim-length` is the truncating spot on the right. `p-jobs-to-start` is the number of CPU cores being used by this command.
- # %%
- %%bash
- qiime deblur denoise-16S \
- --i-demultiplexed-seqs joined-demux.qza \
- --p-trim-length 240 \
- --p-left-trim-len 13 \
- --p-jobs-to-start 4 \
- --o-table deblur-table.qza \
- --o-representative-sequences deblur-rep-seqs.qza \
- --o-stats deblur-denoising-stats.qza
- # %% [markdown]
- # # Process the Feature Table
- # %% [markdown]
- # Now we have obtained feature table and representative sequences, we can further process and clean them.
- #
- # One option is to transform the feature table into relative abundance table. Number of sequence reads may vary greatly across samples/individuals. Transforming the table into relative abundance effectively controls the library size (sum of reads for a sample). Additionally, in some cases microbiome data should be considered compositional and therefore should be represented in percentages.
- # %%
- %%bash
- qiime feature-table relative-frequency \
- --i-table dada2-filtered-table.qza \
- --o-relative-frequency-table dada2-relative-table.qza \
- # %% [markdown]
- # You can also filter by feature prevalence. In the code below, we filter out features whose sum of frequency across all samples is less than 10, and also features present in fewer than 3 samples. You should choose these parameters after inspecting your `table.qzv` file. If you chose to convert your table to relative abundance already, make sure to pass in the relative table in `i-table`, and change `min-frequency` to something like `0.0001`.
- # %%
- %%bash
- qiime feature-table filter-features \
- --i-table dada2-table.qza \
- --p-min-frequency 10 \
- --p-min-samples 3 \
- --o-filtered-table dada2-filtered-table.qza
- # %% [markdown]
- # # Collapsing by Taxa
- # %% [markdown]
- # %% [markdown]
- # If you need to look at the feature table as a tsv (viewable in Excel), you can run the below two commands. The first one converts the `.qza` file to a biom table. Then we can convert the biom table to a tsv.
- # %%
- %%bash
- qiime tools export \
- --input-path dada2-table.qza \
- --output-path dada2-table
- # %%
- %%bash
- biom convert \
- -i dada2-table/feature-table.biom \
- -o dada2-table.tsv --to-tsv
- # %% [markdown]
- # # Assign Taxonomy
- # %% [markdown]
- # Next, we perform taxonomic assignment. QIIME provides classifiers trained on two gene databases: SILVA and Greengenes.
- #
- # SILVA database has a bigger gene tree, is regularly updated, and probably will take more time to run. Greengenes is more compact, was last updated in 2013 (so maybe relatively out-dated), but sometimes it is considered to be more accurate when dealing with specifically human gut microbiome.
- #
- # Below we download two pre-trained Naive Bayes classifiers provided by QIIME. One is trained on SILVA and the other on Greengenes database. Both of these are trained on sequences from only the V4 region in 16S rRNA sequencing. If your sequences are full-length 16S rRNA reads, then get rid of the `-515-806` part of the download link to download classifiers on the full region.
- # %%
- %%bash
- wget \
- -O 'Silva-V4-classifier.qza' \
- 'https://data.qiime2.org/2020.6/common/silva-138-99-515-806-nb-classifier.qza'
- #wget \
- # -O 'Greengenes-V4-classifier.qza' \
- # 'https://data.qiime2.org/2022.2/common/gg-13-8-99-515-806-nb-classifier.qza'
- # %% [markdown]
- # Now we use the classifier to assign taxonomy. The code here uses SILVA database. If you want to use Greengenes, change `i-classifier` to `Greengenes-V4-classifier.qza`.
- # %%
- %%bash
- qiime feature-classifier classify-sklearn \
- --i-classifier Silva-V4-classifier.qza \
- --i-reads dada2-rep-seqs.qza \
- --o-classification taxonomy.qza
- # %% [markdown]
- # # Collapsing by Taxa
- # You can also collapse the feature table to different taxonomic levels, i.e. the columns of your feature table will be genera/families rather than species. `p-level` denotes which taxonomic level you want to collapse to. `7` is the lowest, the species level.
- # %% [markdown]
- # Remember: King Philip came over for good soup
- # %%
- %%bash
- qiime taxa collapse \
- --i-table dada2-table.qza \
- --i-taxonomy taxonomy.qza \
- --p-level 2 \
- --o-collapsed-table phyla-table.qza
- # %% [markdown]
- # After the taxonomic classifier is ready, you can prepare data table for different taxa levels. It is necessary to store both the "filtered" version which has raw counts and the "relative" version which has those counts normalized.
- #
- # 1. Genus Level.
- # %%
- %%bash
- qiime feature-table filter-features \
- --i-table genus-table.qza \
- --p-min-frequency 10 \
- --p-min-samples 3 \
- --o-filtered-table RawCount_Tables/genus-filtered-table.qza
- # %%
- %%bash
- qiime feature-table relative-frequency \
- --i-table RawCount_Tables/genus-filtered-table.qza \
- --o-relative-frequency-table Relative_Tables/genus-relative-table.qza \
- # %%
- %%bash
- qiime tools export \
- --input-path Relative_Tables/genus-relative-table.qza \
- --output-path Relative_Tables/genus-table
- # %%
- %%bash
- biom convert \
- -i Relative_Tables/genus-table/feature-table.biom \
- -o Relative_Tables/genus-table.tsv --to-tsv
- # %%
- %%bash
- qiime tools export \
- --input-path RawCount_Tables/genus-filtered-table.qza \
- --output-path RawCount_Tables/genus-table
- # %%
- %%bash
- biom convert \
- -i RawCount_Tables/genus-table/feature-table.biom \
- -o RawCount_Tables/genus-table.tsv --to-tsv
- # %% [markdown]
- # 2. Family Level
- # %%
- %%bash
- qiime feature-table filter-features \
- --i-table family-table.qza \
- --p-min-frequency 10 \
- --p-min-samples 3 \
- --o-filtered-table RawCount_Tables/family-filtered-table.qza
- # %%
- %%bash
- qiime feature-table relative-frequency \
- --i-table RawCount_Tables/family-filtered-table.qza \
- --o-relative-frequency-table Relative_Tables/family-relative-table.qza \
- # %%
- %%bash
- qiime tools export \
- --input-path Relative_Tables/family-relative-table.qza \
- --output-path Relative_Tables/family-table
- # %%
- %%bash
- biom convert \
- -i Relative_Tables/family-table/feature-table.biom \
- -o Relative_Tables/family-table.tsv --to-tsv
- # %%
- %%bash
- qiime tools export \
- --input-path RawCount_Tables/family-filtered-table.qza \
- --output-path RawCount_Tables/family-table
- # %%
- %%bash
- biom convert \
- -i RawCount_Tables/family-table/feature-table.biom \
- -o RawCount_Tables/family-table.tsv --to-tsv
- # %% [markdown]
- # 3. Order Level
- # %%
- %%bash
- qiime feature-table filter-features \
- --i-table order-table.qza \
- --p-min-frequency 10 \
- --p-min-samples 3 \
- --o-filtered-table RawCount_Tables/order-filtered-table.qza
- # %%
- %%bash
- qiime feature-table relative-frequency \
- --i-table RawCount_Tables/order-filtered-table.qza \
- --o-relative-frequency-table Relative_Tables/order-relative-table.qza \
- # %%
- %%bash
- qiime tools export \
- --input-path Relative_Tables/order-relative-table.qza \
- --output-path Relative_Tables/order-table
- # %%
- %%bash
- biom convert \
- -i Relative_Tables/order-table/feature-table.biom \
- -o Relative_Tables/order-table.tsv --to-tsv
- # %%
- %%bash
- qiime tools export \
- --input-path RawCount_Tables/order-filtered-table.qza \
- --output-path RawCount_Tables/order-table
- # %%
- %%bash
- biom convert \
- -i RawCount_Tables/order-table/feature-table.biom \
- -o RawCount_Tables/order-table.tsv --to-tsv
- # %% [markdown]
- # 4. Species Level
- # %%
- %%bash
- qiime feature-table filter-features \
- --i-table species-table.qza \
- --p-min-frequency 10 \
- --p-min-samples 3 \
- --o-filtered-table RawCount_Tables/species-filtered-table.qza
- # %%
- %%bash
- qiime feature-table relative-frequency \
- --i-table RawCount_Tables/species-filtered-table.qza \
- --o-relative-frequency-table Relative_Tables/species-relative-table.qza \
- # %%
- %%bash
- qiime tools export \
- --input-path Relative_Tables/species-relative-table.qza \
- --output-path Relative_Tables/species-table
- # %%
- %%bash
- biom convert \
- -i Relative_Tables/species-table/feature-table.biom \
- -o Relative_Tables/species-table.tsv --to-tsv
- # %%
- %%bash
- qiime tools export \
- --input-path RawCount_Tables/species-filtered-table.qza \
- --output-path RawCount_Tables/species-table
- # %%
- %%bash
- biom convert \
- -i RawCount_Tables/species-table/feature-table.biom \
- -o RawCount_Tables/species-table.tsv --to-tsv
- # %% [markdown]
- # 5. Phylum Level
- # %%
- %%bash
- qiime feature-table filter-features \
- --i-table phyla-table.qza \
- --p-min-frequency 10 \
- --p-min-samples 3 \
- --o-filtered-table RawCount_Tables/phyla-filtered-table.qza
- # %%
- %%bash
- qiime feature-table relative-frequency \
- --i-table RawCount_Tables/phyla-filtered-table.qza \
- --o-relative-frequency-table Relative_Tables/phyla-relative-table.qza \
- # %%
- %%bash
- qiime tools export \
- --input-path Relative_Tables/phyla-relative-table.qza \
- --output-path Relative_Tables/phyla-table
- # %%
- %%bash
- biom convert \
- -i Relative_Tables/phyla-table/feature-table.biom \
- -o Relative_Tables/phyla-table.tsv --to-tsv
- # %%
- %%bash
- qiime tools export \
- --input-path RawCount_Tables/phyla-filtered-table.qza \
- --output-path RawCount_Tables/phyla-table
- # %%
- %%bash
- biom convert \
- -i RawCount_Tables/phyla-table/feature-table.biom \
- -o RawCount_Tables/phyla-table.tsv --to-tsv
- # %% [markdown]
- # # Create Phylogenetic Tree
- # %% [markdown]
- # We construct a phylogenetic tree based on the representative sequences information. Three algorithms for constructing trees are available in QIIME 2: FastTree, IQ-Tree, and RAxML. FastTree runs the fastest but returns poorest quality. IQ-Tree and RAxML will give somewhat different results, but how exactly they differ varies between datasets. Here the code uses RAxML. If you wish to use any of the other method, simply change `raxml` in the command to `fasttree` or `iqtree`.
- # %%
- %%bash
- qiime phylogeny align-to-tree-mafft-raxml \
- --i-sequences dada2-rep-seqs.qza \
- --p-n-threads 4 \
- --output-dir phylogeny-trees
- # %%
- %%bash
- qiime feature-table filter-seqs \
- --i-data dada2-rep-seqs.qza \
- --i-table dada2-filtered-table.qza \
- --o-filtered-data filtered-rep-seqs.qza
- # %% [markdown]
- # # Calculating Diversity Metrics
- #
- # 1.1 Alpha Diversity (Faith's Phylogenetic Diversity)
- # %%
- %%bash
- qiime diversity alpha-phylogenetic \
- --i-table dada2-table.qza \
- --i-phylogeny phylogeny-trees/rooted_tree.qza \
- --p-metric faith_pd \
- --o-alpha-diversity species-faith-table.qza
- # %%
- %%bash
- qiime tools export \
- --input-path species-faith-table.qza \
- --output-path species-faith-table
- # %%
- %%bash
- biom convert \
- -i species-faith-table/feature-table.biom \
- -o --to-tsv
- # %% [markdown]
- # 1.2 Simpson's Alpha Diversity
- # %%
- %%bash
- qiime diversity alpha \
- --i-table species-filtered-table.qza \
- --p-metric simpson \
- --o-alpha-diversity species-simpson-table.qza
- # %%
- %%bash
- qiime tools export \
- --input-path species-simpson-table.qza \
- --output-path species-simpson-table
- # %%
- %%bash
- biom convert \
- -i species-simpson-table/feature-table.biom \
- -o --to-tsv
- # %% [markdown]
- # 2.1 Beta Diversity (Unweighted UniFrac -- Phylogenetic)
- # %%
- %%bash
- qiime diversity beta-phylogenetic \
- --i-table dada2-table.qza \
- --i-phylogeny phylogeny-trees/rooted_tree.qza \
- --p-metric unweighted_unifrac \
- --o-distance-matrix unweighted_unifrac_distance_matrix.qza
- # %%
- %%bash
- qiime tools export \
- --input-path unweighted_unifrac_distance_matrix.qza \
- --output-path unweighted_unifrac_distance_matrix
- # %% [markdown]
- # 2.11 Processing Beta Diversity Distance Matrix
- # %%
- unifrac_distance_matrix = pd.read_csv("unweighted_unifrac_distance_matrix/distance-matrix.tsv",sep="\t", index_col=0, header=0)
- unifrac_distance_matrix.drop(axis = 0, index=unifrac_distance_matrix.columns[-5:], inplace = True)
- unifrac_distance_matrix.drop(axis = 1, columns=unifrac_distance_matrix.columns[-5:], inplace = True)
- uni_1 = unifrac_distance_matrix.loc[unifrac_distance_matrix.columns.str.startswith("01."),unifrac_distance_matrix.columns.str.startswith("01.")].iloc[:,0]
- uni_4 = unifrac_distance_matrix.loc[unifrac_distance_matrix.columns.str.startswith("04."),unifrac_distance_matrix.columns.str.startswith("04.")].iloc[:,0]
- uni_5 = unifrac_distance_matrix.loc[unifrac_distance_matrix.columns.str.startswith("05."),unifrac_distance_matrix.columns.str.startswith("05.")].iloc[:,0]
- uni_8 = unifrac_distance_matrix.loc[unifrac_distance_matrix.columns.str.startswith("08."),unifrac_distance_matrix.columns.str.startswith("08.")].iloc[:,0]
- uni_9 = unifrac_distance_matrix.loc[unifrac_distance_matrix.columns.str.startswith("09."),unifrac_distance_matrix.columns.str.startswith("09.")].iloc[:,0]
- uni_16 = unifrac_distance_matrix.loc[unifrac_distance_matrix.columns.str.startswith("16."),unifrac_distance_matrix.columns.str.startswith("16.")].iloc[:,0]
- unifrac_distance = pd.concat([uni_1,uni_4, uni_5, uni_8, uni_9, uni_16], axis = 0).to_frame(name = "UniFrac_Distance")
- unifrac_distance.at["09.0808", "UniFrac_Distance"] = (unifrac_distance.at["09.0808.1", "UniFrac_Distance"] + unifrac_distance.at["09.0808.2", "UniFrac_Distance"]) /2
- print(unifrac_distance)
- # %% [markdown]
- # 2.2 Bray Curtis Dissimilarity
- # %%
- species_table = pd.read_csv("Relative_Tables/species-table.tsv", sep = "\t", skiprows=[0], index_col = 0)
- species_table = species_table.T
- species_table.index.names = ["Sample_ID"]
- species_table.drop(species_table.tail(5).index,inplace=True) # drop last n rows
- species = species_table.columns
- def apply_BC(df):
- first_row = df.loc[df.index[0], :].values.flatten().tolist()[0:-2]
- for index in df.index.values:
- row = df.loc[index, :].values.flatten().tolist()[0:-2]
- df.at[index, "BC_Dissimilarity"] = braycurtis(row, first_row)
- return df
- species_table["Player"] = [i.split(".")[0] for i in species_table.index.tolist()]
- species_table["BC_Dissimilarity"] = np.nan
- species_table = species_table.groupby("Player").apply(lambda df: apply_BC(df))
- species_table.index = species_table.index.droplevel(0)
- print(species_table)
- species_table.at["09.0808", "BC_Dissimilarity"] = (species_table.at["09.0808.1", "BC_Dissimilarity"] + species_table.at["09.0808.2", "BC_Dissimilarity"]) /2
- print(species_table.loc[:,"BC_Dissimilarity"])
- # %% [markdown]
- # # Combine Diversity Data, Fecal Sample data, and Survey Data
- # %%
- master_data = pd.read_csv("../Master Spreadsheet - Master Data Sheet.tsv",sep = "\t",dtype={"ID/Study Date ":"str"})
- master_data = master_data.set_index(master_data.columns[0])
- #Load Faith's Diversity
- alpha_diversity_table = pd.read_csv("species-faith-table/alpha-diversity.tsv", sep = "\t",
- index_col=0)
- alpha_diversity_table.drop(alpha_diversity_table.tail(5).index,inplace=True) # drop last n rows
- alpha_diversity_table.at["09.0808", "faith_pd"] = (alpha_diversity_table.at["09.0808.1", "faith_pd"] + alpha_diversity_table.at["09.0808.2", "faith_pd"]) /2
- #Load Simpson's Diversity
- simpson_table = pd.read_csv("species-simpson-table/alpha-diversity.tsv", sep = "\t",
- index_col=0)
- simpson_table.drop(simpson_table.tail(5).index,inplace=True) # drop last n rows
- simpson_table.at["09.0808", "simpson"] = (simpson_table.at["09.0808.1", "simpson"] + simpson_table.at["09.0808.2", "simpson"]) /2
- #Combine all four diveristy metrics
- alpha_diversity_table = alpha_diversity_table.join(unifrac_distance)
- alpha_diversity_table["BC_Dissimilarity"] = species_table["BC_Dissimilarity"]
- alpha_diversity_table = alpha_diversity_table.join(simpson_table)
- #Obtain Days
- # alpha_diversity_table["Player"]= [i.split(".")[0] for i in alpha_diversity_table.index.tolist()]
- # alpha_diversity_table["Date"] = [datetime.datetime.strptime(i.split(".")[1] + "2022", "%m%d%Y").date() for i in alpha_diversity_table.index.tolist()]
- # alpha_diversity_table["Date"] = pd.to_datetime(alpha_diversity_table["Date"])
- # alpha_diversity_table["Days"] = alpha_diversity_table.groupby("Player")["Date"].apply(lambda x: (x - x.iloc[0]).dt.days)
- # print(alpha_diversity_table)
- # alpha_diversity_table["Days"] = [i.days for i in alpha_diversity_table["Days"]]
- alpha_diversity_table["Player"] = [i.split(".")[0] for i in alpha_diversity_table.index.tolist()]
- alpha_diversity_table["Date"] = pd.to_datetime(alpha_diversity_table.index.str.split('.').str[1] + '2022', format='%m%d%Y')
- alpha_diversity_table['Days'] = alpha_diversity_table.groupby('Player')['Date'].transform(lambda x: (x - x.min()).dt.days)
- print(alpha_diversity_table)
- alpha_master_data = master_data.join(alpha_diversity_table)
- alpha_master_data.index.name = "Sample_ID"
- alpha_master_data.to_csv("../diversity_master.tsv", sep ="\t")
- #print(alpha_master_data)
- # %% [markdown]
- # ## Perform same process for different taxa levels
- # %%
- def pct_change_manual(df, colnames):
- n_rows = df.shape[0]
- for i in range(n_rows-1,-1,-1):
- for col in colnames:
- if i ==0:
- df.at[df.index.values[i], col] = np.nan
- elif df.at[df.index.values[i], "TWO FECAL SAMPLES FOLLOWING HEAD IMPACT?"] == 1:
- ##Two days prior
- if np.isnan((df.at[df.index.values[i-2], col])):
- df.at[df.index.values[i], col] = np.nan
- else:
- df.at[df.index.values[i], col] = (df.at[df.index.values[i], col] - df.at[df.index.values[i-2], col])/(df.at[df.index.values[i-2], col] + 0.01)
- else:
- if np.isnan((df.at[df.index.values[i-1], col])):
- df.at[df.index.values[i], col] = np.nan
- else:
- df.at[df.index.values[i], col] = (df.at[df.index.values[i], col] - df.at[df.index.values[i-1], col])/(df.at[df.index.values[i-1], col] + 0.01)
- return df
- # %% [markdown]
- # # Processing taxa data for R -- Relative Data Tables (0-1 normalized)
- # %%
- order_table = pd.read_csv("Relative_Tables/order-table.tsv", sep = "\t", skiprows=[0], index_col = 0)
- order_table = order_table.T
- order_table.index.names = ["Sample_ID"]
- order_table.drop(order_table.tail(5).index,inplace=True) # drop last n rows
- orders = order_table.columns
- for i in range(order_table.shape[1]):
- column_name = order_table.columns[i]
- if "o__" in column_name:
- order_table.rename(columns = {column_name:column_name.split("o__", 1)[1]}, inplace= True)
- order_table.loc["09.0808"] = (order_table.loc["09.0808.1"] + order_table.loc["09.0808.2"]) /2
- order_table.to_csv("../processed_data/order_processed.tsv", sep ="\t")
- genus_table = pd.read_csv("Relative_Tables/genus-table.tsv", sep = "\t", skiprows=[0], index_col = 0)
- genus_table = genus_table.T
- genus_table.index.names = ["Sample_ID"]
- genus_table.drop(genus_table.tail(5).index,inplace=True) # drop last n rows
- for i in range(genus_table.shape[1]):
- column_name = genus_table.columns[i]
- if "g__" in column_name:
- genus_table.rename(columns = {column_name:column_name.split("g__", 1)[1]}, inplace= True)
- genus_table.loc["09.0808"] = (genus_table.loc["09.0808.1"] + genus_table.loc["09.0808.2"]) /2
- genus_table.to_csv("../processed_data/genus_processed.tsv", sep ="\t")
- family_table = pd.read_csv("Relative_Tables/family-table.tsv", sep = "\t", skiprows=[0], index_col = 0)
- family_table = family_table.T
- family_table.index.names = ["Sample_ID"]
- family_table.drop(family_table.tail(5).index,inplace=True) # drop last n rows
- for i in range(family_table.shape[1]):
- column_name = family_table.columns[i]
- if "f__" in column_name:
- family_table.rename(columns = {column_name:column_name.split("f__", 1)[1]}, inplace= True)
- family_table.loc["09.0808"] = (family_table.loc["09.0808.1"] + family_table.loc["09.0808.2"]) /2
- family_table.to_csv("../processed_data/family_processed.tsv", sep ="\t")
- species_table = pd.read_csv("Relative_Tables/species-table.tsv", sep = "\t", skiprows=[0], index_col = 0)
- species_table = species_table.T
- species_table.index.names = ["Sample_ID"]
- species_table.drop(species_table.tail(5).index,inplace=True) # drop last n rows
- for i in range(species_table.shape[1]):
- column_name = species_table.columns[i]
- if "s__" in column_name:
- species_table.rename(columns = {column_name:column_name.split("s__", 1)[1]}, inplace= True)
- species_table.loc["09.0808"] = (species_table.loc["09.0808.1"] + species_table.loc["09.0808.2"]) /2
- species_table.to_csv("../processed_data/species_processed.tsv", sep ="\t")
- # %% [markdown]
- # # Processing Taxa data for R -- Raw Counts (for CLR transform)
- # %%
- order_table = pd.read_csv("RawCount_Tables/order-table.tsv", sep = "\t", skiprows=[0], index_col = 0)
- order_table = order_table.T
- order_table.index.names = ["Sample_ID"]
- order_table.drop(order_table.tail(5).index,inplace=True) # drop last n rows
- orders = order_table.columns
- for i in range(order_table.shape[1]):
- column_name = order_table.columns[i]
- if "o__" in column_name:
- order_table.rename(columns = {column_name:column_name.split("o__", 1)[1]}, inplace= True)
- order_table.loc["09.0808"] = (order_table.loc["09.0808.1"] + order_table.loc["09.0808.2"]) /2
- order_table.to_csv("../processed_data/CLR/order_processed.tsv", sep ="\t")
- genus_table = pd.read_csv("RawCount_Tables/genus-table.tsv", sep = "\t", skiprows=[0], index_col = 0)
- genus_table = genus_table.T
- genus_table.index.names = ["Sample_ID"]
- genus_table.drop(genus_table.tail(5).index,inplace=True) # drop last n rows
- for i in range(genus_table.shape[1]):
- column_name = genus_table.columns[i]
- if "g__" in column_name:
- genus_table.rename(columns = {column_name:column_name.split("g__", 1)[1]}, inplace= True)
- genus_table.loc["09.0808"] = (genus_table.loc["09.0808.1"] + genus_table.loc["09.0808.2"]) /2
- genus_table.to_csv("../processed_data/CLR/genus_processed.tsv", sep ="\t")
- family_table = pd.read_csv("RawCount_Tables/family-table.tsv", sep = "\t", skiprows=[0], index_col = 0)
- family_table = family_table.T
- family_table.index.names = ["Sample_ID"]
- family_table.drop(family_table.tail(5).index,inplace=True) # drop last n rows
- for i in range(family_table.shape[1]):
- column_name = family_table.columns[i]
- if "f__" in column_name:
- family_table.rename(columns = {column_name:column_name.split("f__", 1)[1]}, inplace= True)
- family_table.loc["09.0808"] = (family_table.loc["09.0808.1"] + family_table.loc["09.0808.2"]) /2
- family_table.to_csv("../processed_data/CLR/family_processed.tsv", sep ="\t")
- species_table = pd.read_csv("RawCount_Tables/species-table.tsv", sep = "\t", skiprows=[0], index_col = 0)
- species_table = species_table.T
- species_table.index.names = ["Sample_ID"]
- species_table.drop(species_table.tail(5).index,inplace=True) # drop last n rows
- for i in range(species_table.shape[1]):
- column_name = species_table.columns[i]
- if "s__" in column_name:
- species_table.rename(columns = {column_name:column_name.split("s__", 1)[1]}, inplace= True)
- species_table.loc["09.0808"] = (species_table.loc["09.0808.1"] + species_table.loc["09.0808.2"]) /2
- species_table.to_csv("../processed_data/CLR/species_processed.tsv", sep ="\t")
- # %% [markdown]
- # # Acute Changes after Head Impact
- # %%
- order_master_no_9 = pd.read_csv("../processed_data/order_master_without_9.tsv", sep ="\t", dtype={"Sample_ID":"str"})
- order_master_no_9 = order_master_no_9.set_index(order_master_no_9.columns[0])
- genus_master_no_9 = pd.read_csv("../processed_data/genus_master_without_9.tsv", sep ="\t", dtype={"Sample_ID":"str"})
- genus_master_no_9 = genus_master_no_9.set_index(genus_master_no_9.columns[0])
- family_master_no_9 = pd.read_csv("../processed_data/family_master_without_9.tsv", sep ="\t", dtype={"Sample_ID":"str"})
- family_master_no_9 = family_master_no_9.set_index(family_master_no_9.columns[0])
- species_master_no_9 = pd.read_csv("../processed_data/species_master_without_9.tsv", sep ="\t", dtype={"Sample_ID":"str"})
- species_master_no_9 = species_master_no_9.set_index(species_master_no_9.columns[0])
- # %%
- def __convert_to_block_df(
- a,
- y_col: str = None,
- group_col: str = None,
- block_col: str = None,
- melted: bool = False) -> pd.DataFrame:
- # TODO: refactor conversion of block data to DataFrame
- if melted and not all([i is not None for i in [block_col, group_col, y_col]]):
- raise ValueError(
- '`block_col`, `group_col`, `y_col` should be explicitly specified if using melted data')
- if isinstance(a, pd.DataFrame) and not melted:
- x = a.copy(deep=True)
- group_col = 'groups'
- block_col = 'blocks'
- y_col = 'y'
- x.columns.name = group_col
- x.index.name = block_col
- x = x.reset_index().melt(id_vars=block_col, var_name=group_col, value_name=y_col)
- elif isinstance(a, pd.DataFrame) and melted:
- x = pd.DataFrame.from_dict({'groups': a[group_col],
- 'blocks': a[block_col],
- 'y': a[y_col]})
- elif not isinstance(a, pd.DataFrame):
- x = np.array(a)
- x = pd.DataFrame(x, index=np.arange(
- x.shape[0]), columns=np.arange(x.shape[1]))
- if not melted:
- group_col = 'groups'
- block_col = 'blocks'
- y_col = 'y'
- x.columns.name = group_col
- x.index.name = block_col
- x = x.reset_index().melt(id_vars=block_col, var_name=group_col, value_name=y_col)
- else:
- x.rename(columns={group_col: 'groups',
- block_col: 'blocks', y_col: 'y'}, inplace=True)
- group_col = 'groups'
- block_col = 'blocks'
- y_col = 'y'
- return x, y_col, group_col, block_col
- def posthoc_nemenyi_friedman_q(
- a: Union[list, np.ndarray, pd.DataFrame],
- y_col: str = None,
- block_col: str = None,
- group_col: str = None,
- melted: bool = False,
- sort: bool = False) -> pd.DataFrame:
- '''Calculate pairwise comparisons using Nemenyi post hoc test for
- unreplicated blocked data. This test is usually conducted post hoc if
- significant results of the Friedman's test are obtained. The statistics
- refer to upper quantiles of the studentized range distribution (Tukey) [1]_,
- [2]_, [3]_.
- Parameters
- ----------
- a : array_like or pandas DataFrame object
- An array, any object exposing the array interface or a pandas
- DataFrame.
- If `melted` is set to False (default), `a` is a typical matrix of
- block design, i.e. rows are blocks, and columns are groups. In this
- case you do not need to specify col arguments.
- If `a` is an array and `melted` is set to True,
- y_col, block_col and group_col must specify the indices of columns
- containing elements of correspondary type.
- If `a` is a Pandas DataFrame and `melted` is set to True,
- y_col, block_col and group_col must specify columns names (strings).
- y_col : str or int
- Must be specified if `a` is a pandas DataFrame object.
- Name of the column that contains y data.
- block_col : str or int
- Must be specified if `a` is a pandas DataFrame object.
- Name of the column that contains blocking factor values.
- group_col : str or int
- Must be specified if `a` is a pandas DataFrame object.
- Name of the column that contains treatment (group) factor values.
- melted : bool, optional
- Specifies if data are given as melted columns "y", "blocks", and
- "groups".
- sort : bool, optional
- If True, sort data by block and group columns.
- Returns
- -------
- result : pandas.DataFrame
- P values.
- Notes
- -----
- A one-way ANOVA with repeated measures that is also referred to as ANOVA
- with unreplicated block design can also be conducted via Friedman's
- test. The consequent post hoc pairwise multiple comparison test
- according to Nemenyi is conducted with this function.
- This function does not test for ties.
- References
- ----------
- .. [1] J. Demsar (2006), Statistical comparisons of classifiers over
- multiple data sets, Journal of Machine Learning Research, 7, 1-30.
- .. [2] P. Nemenyi (1963) Distribution-free Multiple Comparisons. Ph.D.
- thesis, Princeton University.
- .. [3] L. Sachs (1997), Angewandte Statistik. Berlin: Springer.
- Pages: 668-675.
- Examples
- --------
- >>> # Non-melted case, x is a block design matrix, i.e. rows are blocks
- >>> # and columns are groups.
- >>> x = np.array([[31,27,24],[31,28,31],[45,29,46],[21,18,48],[42,36,46],[32,17,40]])
- >>> sp.posthoc_nemenyi_friedman(x)
- '''
- def compare_stats(i, j):
- dif = np.abs(R[groups[i]] - R[groups[j]])
- qval = dif / np.sqrt(k * (k + 1.) * 0.975 / (6. * n))
- return qval
- x, _y_col, _group_col, _block_col = __convert_to_block_df(
- a, y_col, group_col, block_col, melted)
- x = x.sort_values(by=[_group_col, _block_col],
- ascending=True) if sort else x
- x.dropna(inplace=True)
- groups = x[_group_col].unique()
- k = groups.size
- n = x[_block_col].unique().size
- x['mat'] = x.groupby(_block_col)[_y_col].rank()
- R = x.groupby(_group_col)['mat'].mean()
- vs = np.zeros((k, k))
- combs = combinations(range(k), 2)
- tri_upper = np.triu_indices(vs.shape[0], 1)
- tri_lower = np.tril_indices(vs.shape[0], -1)
- vs[:, :] = 0
- for i, j in combs:
- vs[i, j] = compare_stats(i, j)
- qs = vs
- return pd.DataFrame(qs, index=groups, columns=groups)
- # %%
- diversity_master_no_9 = pd.read_csv("../diversity_master_without_9.tsv", sep ="\t", dtype={"Sample_ID":"str"})
- diversity_master_no_9 = diversity_master_no_9.set_index(diversity_master_no_9.columns[0])
- #Data Slicing
- def acute_changes(df, metric, axis_label):
- df_new = df
- df_new = df_new[df_new['BC_Dissimilarity'].notna()]
- #threshold = df_new["Impact load sustained 0-24 hours prior"].quantile(0.75)
- #df_new["Sustained_Impact"] = np.where(df_new["Impact load sustained 0-24 hours prior"]> threshold, 1,0)
- threshold = df_new["Impact.load.sustained.0.24.hours.prior"].quantile(0.75)
- print(threshold)
- df_new["Sustained_Impact"] = np.where(df_new["Impact.load.sustained.0.24.hours.prior"]> threshold, 1,0)
- global data_slices
- data_slices = pd.DataFrame(columns=["Day of Impact", "1 day after", "2 days after", "3 days after", "Player"])
- df_new.groupby("Player").apply(lambda df: slice_data(df, metric))
- data_slices_long = data_slices
- data_slices_long["ID"] = data_slices_long.index.values
- data_slices_long = pd.melt(data_slices_long, value_vars=["Day of Impact", "1 day after", "2 days after", "3 days after"],
- var_name="Time", id_vars= ["ID", "Player"], value_name="Value")
- data_slices_long['Player'] = data_slices_long['Player'].astype(str)
- print(data_slices_long)
- boxplot = sns.boxplot(x='Time',y='Value',data=data_slices_long, color = "white", palette="Greys", width=0.6,showfliers = False)
- boxplot = sns.stripplot(x='Time',y='Value',hue='Player',data=data_slices_long, s= 6, jitter =False)
- boxplot = sns.lineplot(
- data=data_slices_long, x="Time", y="Value", units="ID",sort =False,
- color=".6", estimator=None,alpha=0.7
- )
- boxplot.set_xlabel("Hours since Head Impact",fontsize=15)
- boxplot.set_xticklabels(["0-24", "24-48", "48-72", "72-96"])
- boxplot.set_ylabel(axis_label, fontsize=15)
- plt.locator_params(axis='y', nbins=4)
- boxplot.xaxis.set_tick_params(labelsize=12)
- boxplot.yaxis.set_tick_params(labelsize=12)
- handles, previous_labels = boxplot.get_legend_handles_labels()
- lgd = plt.legend(bbox_to_anchor=(1.2, 0.5), loc='center right', borderaxespad=0, title = "Player",
- fontsize="12",handles=handles, labels=["1","4", "5", "8", "9"])
- lgd.get_title().set_fontsize('15')
- boxplot.figure.savefig("../Figures/" + metric + ".png", dpi = 300, bbox_inches = "tight")
- plt.cla()
- data_slices.to_csv(metric + "slices.tsv", sep = "\t")
- (stat, p) = stats.friedmanchisquare(data_slices["Day of Impact"],
- data_slices["1 day after"], data_slices["2 days after"],
- data_slices["3 days after"])
- nemenyi_statistic = posthoc_nemenyi_friedman_q(data_slices[["Day of Impact", "1 day after", "2 days after", "3 days after"]].values)
- with open("../acute_results/" + metric + ".txt", "w") as f:
- print(stat, file= f)
- print(p, file = f)
- print(scikit_posthocs.posthoc_nemenyi_friedman(data_slices.loc[:,["Day of Impact", "1 day after", "2 days after", "3 days after"]]), file = f)
- print("Nemenyi Test Statistic:", nemenyi_statistic)
- def slice_data(df, metric):
- for i in range(1, (df.shape[0]-3)):
- if (df.at[df.index.values[i], "Sustained_Impact"] == 1) & (df.at[df.index.values[i-1], "Sustained_Impact"] == 0):
- #check if days are continuous
- range_dates = [*range(int(df.index.values[i-1].split(".")[1]), int(df.index.values[i-1].split(".")[1])+5 )]
- #print(range_dates)
- actual_dates = [int(k.split(".")[1]) for k in df.index.values[i-1:i+4]]
- #print(actual_dates)
- if np.array_equal(range_dates, actual_dates):
- #if True:
- #check for no impacts in i+1, i+2, i+3, i+4, i+5 days
- if max(df.loc[df.index.values[i+1:i+4], 'Sustained_Impact'] ==0):
- Player = df.index.values[0].split(".")[0]
- #values_to_add = [ df.at[df.index.values[i-1], "simpson"],
- # (df.at[df.index.values[i+1], "simpson"] +df.at[df.index.values[i+2], "simpson"])/2,
- # ((df.at[df.index.values[i+3], "simpson"] +df.at[df.index.values[i+4], "simpson"])/2),
- # df.at[df.index.values[i+5], "simpson"] ]
- values_to_add = [ df.at[df.index.values[i], metric],
- df.at[df.index.values[i+1], metric],df.at[df.index.values[i+2], metric],
- df.at[df.index.values[i+3], metric] , Player]
- data_slices.loc[len(data_slices.index)] = values_to_add
- #print(alpha_master_data)
- acute_changes(diversity_master_no_9, metric="BC_Dissimilarity", axis_label="Bray Curtis Dissimilarity")
- acute_changes(diversity_master_no_9, metric="faith_pd", axis_label="Faith's Phylogenetic Diversity")
- #acute_changes(alpha_master_data, metric="simpson", axis_label="Simpson's Diversity")
- #acute_changes(alpha_master_data, metric="UniFrac_Distance", axis_label="Unifrac Distance")
- # acute_changes(df=order_master_no_9, metric = "Bacteroidales", axis_label = "Bacteroidales")
- # acute_changes(df=order_master_no_9, metric = "Bifidobacteriales", axis_label = "Bifidobacteriales")
- # acute_changes(df=order_master_no_9, metric = "Lactobacillales", axis_label = "Lactobacillales")
- # acute_changes(df=order_master_no_9, metric = "Burkholderiales", axis_label ="Burkholderiales")
- # acute_changes(df=order_master_no_9, metric = "Enterobacterales", axis_label= "Enterobacterales")
- # acute_changes(df=order_master_no_9, metric = "Pseudomonadales", axis_label = "Pseudomonadales")
- # acute_changes(df=order_master_no_9, metric = "Campylobacterales", axis_label = "Campylobacterales")
- # acute_changes(df=order_master_no_9, metric = "Clostridiales", axis_label = "Clostridiales")
- # acute_changes(df=order_master_no_9, metric = "Verrucomicrobiales", axis_label = "Verrucomicrobiales")
- # acute_changes(df=family_master_no_9, metric = "Micrococcaceae", axis_label = "Micrococcaceae")
- # acute_changes(df=family_master_no_9, metric = "Prevotellaceae", axis_label = "Prevotellaceae")
- # acute_changes(df=family_master_no_9, metric = "Leuconostocaceae", axis_label = "Leuconostocaceae")
- # acute_changes(df=family_master_no_9, metric = "Streptococcaceae", axis_label = "Streptococcaceae")
- # acute_changes(df=family_master_no_9, metric = "Peptococcaceae", axis_label = "Peptococcaceae")
- # acute_changes(df=family_master_no_9, metric = "Lachnospiraceae", axis_label= "Lachnospiraceae")
- # acute_changes(df=family_master_no_9, metric = "Bacteroidaceae", axis_label = "Bacteroidaceae")
- # acute_changes(df=family_master_no_9, metric = "Rikenellaceae", axis_label = "Rikenellaceae")
- # acute_changes(df=family_master_no_9, metric = "Ruminococcaceae", axis_label = "Ruminococcaceae")
- # acute_changes(df=family_master_no_9, metric = "Clostridiaceae", axis_label = "Clostridiaceae")
- # acute_changes(df=family_master_no_9, metric = "Bifidobacteriaceae", axis_label ="Bifidobacteriaceae")
- # acute_changes(df=genus_master_no_9, metric = "Prevotella", axis_label = "Prevotella")
- # acute_changes(df=genus_master_no_9, metric = "Weissella", axis_label = "Weissella")
- # acute_changes(df=genus_master_no_9, metric = "Lactococcus", axis_label = "Lactococcus")
- # acute_changes(df=genus_master_no_9, metric = "Anaerostipes", axis_label = "Anaerostipes")
- # acute_changes(df=genus_master_no_9, metric = "Lactobacillus", axis_label = "Lactobacillus")
- # acute_changes(df=genus_master_no_9, metric = "Ruminococcus", axis_label = "Ruminococcus")
- # acute_changes(df=genus_master_no_9, metric = "Bifidobacterium", axis_label = "Bifidobacterium")
- # acute_changes(df=genus_master_no_9, metric = "Bacteroides", axis_label = "Bacteroides")
- # acute_changes(df=species_master_no_9, metric = "Anaerostipes_hadrus", axis_label = "Anaerostipes_hadrus")
- # %% [markdown]
- # # Pre Mid Post Analysis
- # %%
- # Pre Mid Post Analysis
- def perform_pmp(dataset, colname, axislabel, skip_first = True):
- global pre_mid_post
- pre_mid_post = pd.DataFrame(columns=["Pre", "Mid", "Post", "Player"])
- dataset.loc[:,[colname, "Player"] ].groupby("Player").apply(lambda x: find_pre_mid_post(x, colname, skip_first) )
- pre_mid_post = pre_mid_post.loc[3:,["Pre", "Mid", "Post", "Player"]]
- print(pre_mid_post)
- pre_mid_post.to_csv(colname + "pre_mid_post.tsv", sep = "\t")
- pre_mid_post_long = pre_mid_post
- pre_mid_post_long["ID"] = pre_mid_post_long.index.values
- pre_mid_post_long = pd.melt(pre_mid_post_long, value_vars=["Pre","Mid","Post"],
- var_name="Time", id_vars= ["ID", "Player"], value_name="Value")
- print(pre_mid_post_long)
- pre_mid_post_long['Player'] = pre_mid_post_long['Player'].astype(str)
- boxplot = sns.boxplot(x='Time',y='Value',data=pre_mid_post_long, palette="Greys", width=0.5, showfliers=False)
- boxplot = sns.stripplot(x='Time',y='Value',hue='Player',data=pre_mid_post_long, s= 6, jitter =False)
- boxplot = sns.lineplot(
- data=pre_mid_post_long, x="Time", y="Value", units="ID",sort =False,
- color=".7", estimator=None,alpha=0.7
- )
- boxplot.set_xlabel("Collection Period",fontsize=15)
- boxplot.set_xticklabels(["Early", "Middle", "Late"])
- boxplot.set_ylabel(axislabel, fontsize=15)
- plt.locator_params(axis='y', nbins=4)
- boxplot.xaxis.set_tick_params(labelsize=12)
- boxplot.yaxis.set_tick_params(labelsize=12)
- handles, previous_labels = boxplot.get_legend_handles_labels()
- lgd = plt.legend(bbox_to_anchor=(1.2, 0.5), loc='center right', borderaxespad=0, title = "Player",
- fontsize="12", handles = handles, labels = ["1","4","5","8","9","16"])
- lgd.get_title().set_fontsize('15')
- boxplot.figure.savefig("../PreMidPost_Figures/" + colname + "_pre_mid_post.png", dpi =300, bbox_inches = "tight")
- plt.cla()
- #(stat, p) = stats.kruskal(pre_mid_post_BC["Pre"], pre_mid_post_BC["Mid"],pre_mid_post_BC["Post"])
- #print(stat)
- #print(p)
- nemenyi_statistic = posthoc_nemenyi_friedman_q(pre_mid_post.loc[:, ["Pre", "Mid", "Post"]])
- with open("../PreMidPost/" + colname + ".txt", "w") as f:
- (stat, p) = stats.friedmanchisquare( pre_mid_post["Pre"], pre_mid_post["Mid"],pre_mid_post["Post"])
- print(stat, file =f)
- print(p, file = f)
- print(scikit_posthocs.posthoc_nemenyi_friedman(pre_mid_post.loc[:, ["Pre", "Mid", "Post"]]), file =f)
- print("Nemenyi Test Statistic:", nemenyi_statistic)
- def find_pre_mid_post(df, colname, skip_first):
- nrows = df.shape[0]
- center = math.floor(nrows/2)
- #pre_mid_post.loc[len(pre_mid_post)] = [df.loc[df.index.values[1:4], colname].mean(), df.loc[df.index.values[center-1:center+2], colname].mean(),df.loc[df.index.values[nrows-3:nrows], colname].mean()]
- #pre_mid_post_BC.loc[len(pre_mid_post_BC)] = [df.at[df.index.values[0], colname], df.at[df.index.values[center-1], colname],df.at[df.index.values[nrows-4], colname]]
- Player = df.index.values[0].split(".")[0]
- if skip_first:
- if (df.index.values[0].startswith("01.")| df.index.values[0].startswith("04.")):
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[1], colname], df.at[df.index.values[center-1], colname],df.at[df.index.values[nrows-5], colname], Player]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[2], colname], df.at[df.index.values[center], colname],df.at[df.index.values[nrows-4], colname], Player]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[3], colname], df.at[df.index.values[center+1], colname],df.at[df.index.values[nrows-3], colname], Player]
- else:
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[1], colname], df.at[df.index.values[center-1], colname],df.at[df.index.values[nrows-3], colname], Player]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[2], colname], df.at[df.index.values[center], colname],df.at[df.index.values[nrows-2], colname], Player]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[3], colname], df.at[df.index.values[center+1], colname],df.at[df.index.values[nrows-1], colname], Player]
- else:
- if (df.index.values[0].startswith("01.")| df.index.values[0].startswith("04.")):
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[0], colname], df.at[df.index.values[center-1], colname],df.at[df.index.values[nrows-5], colname], Player]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[1], colname], df.at[df.index.values[center], colname],df.at[df.index.values[nrows-4], colname], Player]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[2], colname], df.at[df.index.values[center+1], colname],df.at[df.index.values[nrows-3], colname], Player]
- else:
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[0], colname], df.at[df.index.values[center-1], colname],df.at[df.index.values[nrows-3], colname], Player]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[1], colname], df.at[df.index.values[center], colname],df.at[df.index.values[nrows-2], colname], Player]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[2], colname], df.at[df.index.values[center+1], colname],df.at[df.index.values[nrows-1], colname], Player]
- perform_pmp(diversity_master_no_9, "BC_Dissimilarity", "Bray Curtis Dissimilarity", True)
- perform_pmp(diversity_master_no_9, "faith_pd", "Faith's Phylogenetic Diversity", True)
- # acute_changes(df=order_master_no_9, metric = "Bacteroidales", axis_label = "Bacteroidales")
- # acute_changes(df=order_master_no_9, metric = "Bifidobacteriales", axis_label = "Bifidobacteriales")
- # acute_changes(df=order_master_no_9, metric = "Lactobacillales", axis_label = "Lactobacillales")
- ##perform_pmp(order_master_no_9, "Coriobacteriales", "Coriobacteriales", False)
- ##perform_pmp(order_master_no_9, "Burkholderiales", "Burkholderiales", False)
- # acute_changes(df=order_master_no_9, metric = "Enterobacterales", axis_label= "Enterobacterales")
- # acute_changes(df=order_master_no_9, metric = "Pseudomonadales", axis_label = "Pseudomonadales")
- # acute_changes(df=order_master_no_9, metric = "Campylobacterales", axis_label = "Campylobacterales")
- # acute_changes(df=order_master_no_9, metric = "Clostridiales", axis_label = "Clostridiales")
- # acute_changes(df=order_master_no_9, metric = "Verrucomicrobiales", axis_label = "Verrucomicrobiales")
- # acute_changes(df=family_master_no_9, metric = "Micrococcaceae", axis_label = "Micrococcaceae")
- ##perform_pmp(family_master_no_9, "Prevotellaceae", "Prevotellaceae", False)
- # acute_changes(df=family_master_no_9, metric = "Leuconostocaceae", axis_label = "Leuconostocaceae")
- # acute_changes(df=family_master_no_9, metric = "Streptococcaceae", axis_label = "Streptococcaceae")
- ##perform_pmp(family_master_no_9, "Peptococcaceae", "Peptococcaceae", False)
- ##perform_pmp(family_master_no_9, "Lachnospiraceae", "Lachnospiraceae", False)
- # acute_changes(df=family_master_no_9, metric = "Bacteroidaceae", axis_label = "Bacteroidaceae")
- # acute_changes(df=family_master_no_9, metric = "Rikenellaceae", axis_label = "Rikenellaceae")
- ##perform_pmp(family_master_no_9, "Ruminococcaceae", "Ruminococcaceae")
- # acute_changes(df=family_master_no_9, metric = "Clostridiaceae", axis_label = "Clostridiaceae")
- # acute_changes(df=family_master_no_9, metric = "Bifidobacteriaceae", axis_label ="Bifidobacteriaceae")
- ##perform_pmp(genus_master_no_9, "Prevotella", "Prevotella", False)
- ##perform_pmp(genus_master_no_9, "Weissella", "Weissella", False)
- # acute_changes(df=genus_master_no_9, metric = "Lactococcus", axis_label = "Lactococcus")
- # acute_changes(df=genus_master_no_9, metric = "Anaerostipes", axis_label = "Anaerostipes")
- # acute_changes(df=genus_master_no_9, metric = "Lactobacillus", axis_label = "Lactobacillus")
- ##perform_pmp(genus_master_no_9, "Ruminococcus", "Ruminococcus", False)
- # acute_changes(df=genus_master_no_9, metric = "Bifidobacterium", axis_label = "Bifidobacterium")
- # acute_changes(df=genus_master_no_9, metric = "Bacteroides", axis_label = "Bacteroides")
- # %% [markdown]
- # # Phyla Level Analysis
- # %%
- phyla_table = pd.read_csv("phyla-table.tsv", sep = "\t", skiprows=[0], index_col = 0)
- phyla_table = phyla_table.T
- phyla_table.index.names = ["Sample_ID"]
- #phyla_table.drop(phyla_table.tail(5).index,inplace=True) # drop last n rows
- for i in range(phyla_table.shape[1]):
- column_name = phyla_table.columns[i]
- if "p__" in column_name:
- phyla_table.rename(columns = {column_name:column_name.split("p__", 1)[1]}, inplace= True)
- phyla_table.loc["09.0808"] = (phyla_table.loc["09.0808.1"] + phyla_table.loc["09.0808.2"]) /2
- phyla_table.rename(columns={'d__Bacteria;__': 'Unclassified'}, inplace=True)
- phyla_table.to_csv("../phyla_processed_with_control.tsv", sep ="\t")
- # %%
- phyla = sorted(phyla_table.columns.values.tolist())
- print(phyla)
- phyla_master_no_9 = pd.read_csv("../phyla_master_without_9.tsv", sep ="\t", dtype={"Sample_ID":"str"})
- phyla_master_no_9 = phyla_master_no_9.set_index(phyla_master_no_9.columns[0])
- pre_phyla = pd.DataFrame(columns=np.append(phyla, ["Player"]))
- mid_phyla = pd.DataFrame(columns=np.append(phyla, ["Player"]))
- post_phyla = pd.DataFrame(columns=np.append(phyla, ["Player"]))
- def find_pre_mid_post_phyla(df, list_of_phyla):
- global pre_phyla
- global mid_phyla
- global post_phyla
- nrows = df.shape[0]
- center = math.floor(nrows/2)
- Player = df.index.values[0].split(".")[0]
- if (df.index.values[0].startswith("01.")| df.index.values[0].startswith("04.")):
- pre_phyla = pre_phyla.append(pd.DataFrame(df.loc[df.index.values[1:4],phyla].mean()).T.assign(Player = Player), ignore_index=True)
- mid_phyla = mid_phyla.append(pd.DataFrame(df.loc[df.index.values[center-1:center+2],phyla].mean()).T.assign(Player=Player), ignore_index=True)
- post_phyla = post_phyla.append(pd.DataFrame(df.loc[df.index.values[nrows-5:nrows-2],phyla].mean()).T.assign(Player = Player), ignore_index=True)
- else:
- pre_phyla = pre_phyla.append(pd.DataFrame(df.loc[df.index.values[1:4],phyla].mean()).T.assign(Player = Player), ignore_index=True)
- mid_phyla = mid_phyla.append(pd.DataFrame(df.loc[df.index.values[center-1:center+2],phyla].mean()).T.assign(Player=Player), ignore_index=True)
- post_phyla = post_phyla.append(pd.DataFrame(df.loc[df.index.values[nrows-3:nrows],phyla].mean()).T.assign(Player = Player), ignore_index=True)
- phyla_master_no_9.groupby("Player").apply(lambda x: find_pre_mid_post_phyla(x, phyla) )
- find_pre_mid_post_phyla(phyla_master_no_9, phyla)
- pre_phyla.drop(index = pre_phyla.head(1).index,inplace=True)
- pre_phyla.drop(index = pre_phyla.tail(1).index,inplace=True)
- mid_phyla.drop(index=mid_phyla.head(1).index,inplace=True)
- mid_phyla.drop(index = mid_phyla.tail(1).index,inplace=True)
- post_phyla.drop(index =post_phyla.head(1).index,inplace=True)
- post_phyla.drop(index =post_phyla.tail(1).index,inplace=True)
- fig, ax = plt.subplots(3,6)
- for i,time in enumerate(["Pre", "Mid", "Post"]):
- player_title = ["1", "4", "5", "8", "9", "16"]
- time_label = ["Early", "Middle", "Late"]
- for j, Player in enumerate(["01", "04", "05", "08", "09", "16"]):
- if time == "Pre":
- fracs = pre_phyla.loc[pre_phyla['Player'] == Player, phyla]
- elif time == "Mid":
- fracs = mid_phyla.loc[mid_phyla['Player'] == Player, phyla]
- else:
- fracs = post_phyla.loc[post_phyla['Player'] == Player, phyla]
- ax[i, j].text(0.5, -0.2, player_title[j], transform=ax[i, j].transAxes,
- horizontalalignment='center', verticalalignment='center',
- fontsize =15)
- #ax[i,j].set_title(player_title[j], y=-0.01)
- ax[i,j].pie(fracs, labels=None,
- autopct=None, shadow=False, startangle=90,
- colors=["blue","orange","red", "cyan","pink","purple","green","grey","brown"])
- ax[i,j].set_aspect('equal')
- # label y axis
- if ax[i,j].is_first_col():
- ax[i,j].set_ylabel(time_label[i], fontsize = 15, rotation = 0,labelpad=18)
- fig.text(0.55, -0.01, "Player", ha="center", fontsize=15)
- plt.tight_layout(pad=0.4, w_pad=0.5, h_pad=1.0)
- lgd = fig.legend(phyla, loc = 'center right',bbox_to_anchor=(1.3, 0.5))
- fig.savefig("../PreMidPost_Figures/PreMidPost_phyla.png", dpi = 300, bbox_inches = "tight")
- plt.close()
- ### Stacked Bar Plot
- fig, ax = plt.subplots(3,1, sharex = True)
- for i,time in enumerate(["Early", "Middle", "Late"]):
- if time == "Early":
- fracs = pre_phyla
- elif time == "Middle":
- fracs = mid_phyla
- else:
- fracs = post_phyla
- fracs.plot.bar(x = "Player", stacked = True, ax=ax[i], legend = False,
- color=["blue","orange","red", "cyan","pink","purple","green","grey","brown"])
- ax[i].set_title(time, fontsize = 15, loc='left')
- ax[i].yaxis.set_tick_params(labelsize=12)
- ax[1].set_ylabel('Relative Abundance', fontsize = 15)
- ax[2].set_xlabel("Player", fontsize =15)
- ax[2].set_xticklabels(["1", "4", "5", "8", "9", "16"], rotation = 0, fontsize =12)
- plt.tight_layout(pad=0.4, w_pad=0.6, h_pad=1.0)
- plt.subplots_adjust(wspace=0, hspace=0.5)
- lgd = fig.legend(phyla, loc = 'center right',bbox_to_anchor=(1.3, 0.5), borderaxespad = 0., title= 'Phyla')
- lgd.get_title().set_fontsize('15')
- fig.savefig("../PreMidPost_Figures/PreMidPost_phyla_bar.png", dpi = 300, bbox_inches = "tight")
- plt.close()
- # Adding metadata for PDF file
- # %%
- phyla_master_no_9_control = pd.read_csv("../phyla_master_without_9_with_control.tsv", sep ="\t", dtype={"Sample_ID":"str"})
- phyla_master_no_9_control = phyla_master_no_9_control.set_index(phyla_master_no_9_control.columns[0])
- phyla_master_no_9_control_players = phyla_master_no_9_control.drop(
- phyla_master_no_9_control.tail(5).index) # drop last n rows
- phyla_master_no_9_control_players["Player"] = [i.split(".")[0] for i in phyla_master_no_9_control_players.index.tolist()]
- first_data_points = phyla_master_no_9_control_players.groupby("Player").first().reset_index().loc[:,np.append(phyla, ["Player"])]
- controls = phyla_master_no_9_control.tail(5)
- controls.sort_index(axis=1, inplace=True)
- controls["Player"] = controls.index.values
- print(controls)
- fig, ax = plt.subplots(1,2, sharey = True)
- first_data_points.plot.bar(x = "Player", stacked = True, ax=ax[0], legend = False,
- color=["blue","orange","red", "brown","pink","purple","green","grey","cyan"])
- controls.plot.bar(x = "Player", stacked = True, ax=ax[1], legend = False,
- color=["blue","orange","red", "brown","pink","purple","green","grey","cyan"])
- ax[0].set_ylabel('Relative Abundance', fontsize = 15)
- ax[0].set_xlabel("Player", fontsize =15)
- ax[1].set_xlabel("Control Samples", fontsize =15)
- ax[0].set_xticklabels(["1", "4", "5", "8", "9", "16"], rotation = 0, fontsize =12)
- ax[1].set_xticklabels(["Mock", "N1", "N2", "P1", "P2"], rotation = 0, fontsize =12)
- ax[0].yaxis.set_tick_params(labelsize=12)
- plt.tight_layout(pad=0.4, w_pad=0.6, h_pad=1.0)
- plt.subplots_adjust(wspace=0.1, hspace=0)
- plt.locator_params(axis='y', nbins=3)
- lgd = fig.legend(phyla, loc = 'center right',bbox_to_anchor=(1.3, 0.5), borderaxespad = 0., title= 'Phyla')
- lgd.get_title().set_fontsize('15')
- fig.savefig("../PreMidPost_Figures/Control_Bar.png", dpi = 300, bbox_inches = "tight")
- plt.close()
- # %%
- phyla = phyla_table.columns
- phyla_master_no_9 = pd.read_csv("../phyla_master_without_9.tsv", sep ="\t", dtype={"Sample_ID":"str"})
- phyla_master_no_9 = phyla_master_no_9.set_index(phyla_master_no_9.columns[0])
- pre_mid_post_phyla = pd.DataFrame(columns=["Pre", "Mid", "Post", "Player", "Phyla"])
- phyla_master_no_9.loc[:,[phyla, "Player"] ].groupby("Player").apply(lambda x: find_pre_mid_post(x, colname, skip_first) )
- pre_mid_post = pre_mid_post.loc[3:,["Pre", "Mid", "Post", "Player", "Phyla"]]
- pre_mid_post_long = pre_mid_post
- pre_mid_post_long["ID"] = pre_mid_post_long.index.values
- pre_mid_post_long = pd.melt(pre_mid_post_long, value_vars=["Pre","Mid","Post"],
- var_name="Time", id_vars= ["ID", "Player"], value_name="Value")
- print(pre_mid_post_long)
- pre_mid_post_long['Player'] = pre_mid_post_long['Player'].astype(str)
- def find_pre_mid_post(df, colname, skip_first):
- nrows = df.shape[0]
- center = math.floor(nrows/2)
- #pre_mid_post.loc[len(pre_mid_post)] = [df.loc[df.index.values[1:4], colname].mean(), df.loc[df.index.values[center-1:center+2], colname].mean(),df.loc[df.index.values[nrows-3:nrows], colname].mean()]
- #pre_mid_post_BC.loc[len(pre_mid_post_BC)] = [df.at[df.index.values[0], colname], df.at[df.index.values[center-1], colname],df.at[df.index.values[nrows-4], colname]]
- Player = df.index.values[0].split(".")[0]
- if (df.index.values[0].startswith("01.")| df.index.values[0].startswith("04.")):
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[0], colname], df.at[df.index.values[center-1], colname],df.at[df.index.values[nrows-5], colname], Player]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[1], colname], df.at[df.index.values[center], colname],df.at[df.index.values[nrows-4], colname], Player]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[2], colname], df.at[df.index.values[center+1], colname],df.at[df.index.values[nrows-3], colname], Player]
- else:
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[0], colname], df.at[df.index.values[center-1], colname],df.at[df.index.values[nrows-3], colname], Player]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[1], colname], df.at[df.index.values[center], colname],df.at[df.index.values[nrows-2], colname], Player]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[2], colname], df.at[df.index.values[center+1], colname],df.at[df.index.values[nrows-1], colname], Player]
- # %%
- pre_mid_post = pd.DataFrame(columns=["Pre", "Mid", "Post"])
- def find_pre_mid_post(df, colname):
- global pre_mid_post
- nrows = df.shape[0]
- center = math.floor(nrows/2)
- #pre_mid_post.loc[len(pre_mid_post)] = [df.loc[df.index.values[1:4], colname].mean(), df.loc[df.index.values[center-1:center+2], colname].mean(),df.loc[df.index.values[nrows-3:nrows], colname].mean()]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[0], colname], df.at[df.index.values[center-1], colname],df.at[df.index.values[nrows-4], colname]]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[1], colname], df.at[df.index.values[center], colname],df.at[df.index.values[nrows-3], colname]]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[2], colname], df.at[df.index.values[center+1], colname],df.at[df.index.values[nrows-2], colname]]
- pre_mid_post.loc[len(pre_mid_post)] = [df.at[df.index.values[3], colname], df.at[df.index.values[center+2], colname],df.at[df.index.values[nrows-1], colname]]
- #print(alpha_diversity_master_dummy)
- alpha_diversity_master_dummy.loc[:,["faith_pd", "Player"] ].groupby("Player").apply(lambda x: find_pre_mid_post(x, "faith_pd") )
- pre_mid_post = pre_mid_post.loc[4:,["Pre", "Mid", "Post"]]
- print(pre_mid_post)
- pre_mid_post.boxplot().figure.savefig("alpha_pre_mid_post.png", dpi =300, bbox_inches="tight")
- (stat, p) = stats.f_oneway(pre_mid_post["Pre"], pre_mid_post["Mid"],pre_mid_post["Post"])
- print(stat)
- print(p)
- (stat, p) = stats.friedmanchisquare( pre_mid_post["Pre"], pre_mid_post["Mid"],pre_mid_post["Post"])
- print(stat)
- print(p)
- print(stats.wilcoxon(pre_mid_post["Pre"], pre_mid_post["Post"]))
- # %% [markdown]
- # # Metabolomics Analysis
- # %%
- %%bash
- ../FAPROTAX_1.2.6/collapse_table.py \
- -i species-table/feature-table.biom \
- -o func_table.tsv \
- -g ../FAPROTAX_1.2.6/FAPROTAX.txt -v
- # %%
- func_table = pd.read_csv("func_table.tsv", sep = "\t", index_col = 0)
- func_table = func_table.T
- func_table.index.names = ["Sample_ID"]
- func_table.drop(func_table.tail(5).index,inplace=True) # drop last n rows
- func_table.loc["09.0808"] = (func_table.loc["09.0808.1"] + func_table.loc["09.0808.2"]) /2
- func_table.to_csv("../func_processed.tsv",sep ="\t")
- # %% [markdown]
- # # PCoA Code
- # %%
- df_new = species_master_no_9
- threshold = df_new["Impact.load.sustained.0.24.hours.prior"].quantile(0.75)
- df_new["Sustained_Impact"] = np.where(df_new["Impact.load.sustained.0.24.hours.prior"] > threshold, 1, 0)
- species = species_master_no_9.columns.tolist()
- species = species[0:202]
- #print(species)
- sliced_df = pd.DataFrame(columns=np.append(species, ["Player"]))
- def slice_data_for_PCoA(group_df):
- result_df = pd.DataFrame(columns=sliced_df.columns)
- for i in range(1, (group_df.shape[0] - 3)):
- if (group_df.at[group_df.index.values[i], "Sustained_Impact"] == 1) & (group_df.at[group_df.index.values[i - 1], "Sustained_Impact"] == 0):
- # check if days are continuous
- range_dates = [*range(int(group_df.index.values[i - 1].split(".")[1]), int(group_df.index.values[i - 1].split(".")[1]) + 5)]
- actual_dates = [int(k.split(".")[1]) for k in group_df.index.values[i - 1:i + 4]]
- if np.array_equal(range_dates, actual_dates):
- # check for no impacts in i+1, i+2, i+3, i+4, i+5 days
- if max(group_df.loc[group_df.index.values[i + 1:i + 4], 'Sustained_Impact'] == 0):
- Player = group_df.index.values[0].split(".")[0]
- result_df = result_df.append(group_df.loc[group_df.index.values[i - 1:i + 4], species + ["Player"]].assign(Player=Player), ignore_index = True)
- #print(result_df)
- return result_df
- grouped = df_new.groupby("Player")
- sliced_df = pd.concat([slice_data_for_PCoA(group_df) for _, group_df in grouped], ignore_index = True)
- #sliced_df=sliced_df.loc[:,species]
- print(sliced_df)
- #print(pdist(sliced_df))
- matrix = squareform(pdist(sliced_df, metric = "euclidean"))
- print(matrix)
- pcoa_results = pcoa(matrix)
- pcoa_df = pcoa_results.samples[['PC1', 'PC2']]
- print(pcoa_df.shape)
- a = np.array(["Day Pre", "Day of Impact", "1 day after", "2 days after", "3 days after"])
- pcoa_df["Day"] = np.tile(a,int(sliced_df.shape[0]/5))
- plt.scatter(pcoa_df.loc[pcoa_df["Day"] == "Day Pre", "PC1"],
- pcoa_df.loc[pcoa_df["Day"] == "Day Pre", "PC2"],c = 'black')
- plt.scatter(pcoa_df.loc[pcoa_df["Day"] == "Day of Impact", "PC1"],
- pcoa_df.loc[pcoa_df["Day"] == "Day of Impact", "PC2"],c = 'blue')
- plt.scatter(pcoa_df.loc[pcoa_df["Day"] == "1 day after", "PC1"],
- pcoa_df.loc[pcoa_df["Day"] == "1 day after", "PC2"],c = 'red')
- plt.scatter(pcoa_df.loc[pcoa_df["Day"] == "2 days after", "PC1"],
- pcoa_df.loc[pcoa_df["Day"] == "2 days after", "PC2"],c = 'green')
- plt.scatter(pcoa_df.loc[pcoa_df["Day"] == "3 days after", "PC1"],
- pcoa_df.loc[pcoa_df["Day"] == "3 days after", "PC2"],c = 'purple')
- def encircle2(x,y, ax=None, **kw):
- if not ax: ax=plt.gca()
- p = np.c_[x,y]
- mean = np.mean(p, axis=0)
- d = p-mean
- r = np.max(np.sqrt(d[:,0]**2+d[:,1]**2 ))
- circ = plt.Circle(mean, radius=1.05*r,**kw)
- ax.add_patch(circ)
- encircle2(pcoa_df.loc[pcoa_df["Day"] == "Day Pre", "PC1"],
- pcoa_df.loc[pcoa_df["Day"] == "Day Pre", "PC2"], ec="black", fc="none")
- encircle2( pcoa_df.loc[pcoa_df["Day"] == "Day of Impact", "PC1"],
- pcoa_df.loc[pcoa_df["Day"] == "Day of Impact", "PC2"], ec = 'blue',fc="none")
- encircle2(pcoa_df.loc[pcoa_df["Day"] == "1 day after", "PC1"],
- pcoa_df.loc[pcoa_df["Day"] == "1 day after", "PC2"],ec = 'red', fc = "none")
- encircle2(pcoa_df.loc[pcoa_df["Day"] == "2 days after", "PC1"],
- pcoa_df.loc[pcoa_df["Day"] == "2 days after", "PC2"],ec = 'green', fc = "none")
- encircle2(pcoa_df.loc[pcoa_df["Day"] == "3 days after", "PC1"],
- pcoa_df.loc[pcoa_df["Day"] == "3 days after", "PC2"],ec = 'purple', fc="none")
- plt.gca().relim()
- plt.gca().autoscale_view()
- plt.xlabel("PC1")
- plt.ylabel("PC2")
- plt.legend(["Day Pre", "Day of Impact", "1 day after", "2 days after", "3 days after"])
- plt.savefig("PcoA_01.png", dpi = 300, bbox_inches="tight")
TBI Microbiome.ipynb at commit d13d3d7, no license · at the source
Overview
- Program in Neuroscience, Colgate University, Hamilton, New York, United States of America
- Department of Biology, Colgate University, Hamilton, New York, United States of America
- Department of Mathematics, Colgate University, Hamilton, New York, United States of America
Abstract
Non-concussive head impacts (NHIs) are a significant health concern among at-risk groups, including athletes and military personnel. NHIs are hits to the head or head acceleration events (HAEs) that do not generate clinically detectable symptoms and are unlikely to meet diagnostic criteria for mild traumatic brain injury (mTBI). The composition of the gut microbiota influences many aspects of health and wellness and can be altered by TBIs and by brain-related diseases and disorders; however, microbiome alterations have not previously been linked to NHIs. We investigated whether NHIs in a cohort of American football players correlate with acute and long-term changes in the gut microbiome. This study monitored head impact exposure, gut microbiome composition, and a breadth of clinical and behavioral factors in a cohort of collegiate American football players across a competition season. Both short- and long-term changes in the microbiome were analyzed for correlation with head impact events and mathematical modeling was used to examine the contribution of NHIs and other clinical factors to these changes. We observe that NHI exposure correlates with changes in microbial diversity and composition three days following a head impact event. Furthermore, the athletes’ gut microbiomes change significantly across the season, with evidence from mixed-effects modeling indicating that the cumulative effects of NHIs contribute to this change. Our results provide strong evidence for a link between NHIs and changes in the diversity and composition of the gut microbiome. The outcomes of this study emphasize the importance of careful monitoring of head impacts, including those that do not generate clinical symptoms.
Reproduced under the paper's license (CC BY), from the paper cited above.
Repository
Its files are read in the Code ↔ Paper reader above, with 12 matches between paragraphs and lines of code.
aziz-zafar/TBI-Microbiome
d13d3d7c12fa1833180f138096ba1848f150c08e, 27 January 2026Availability: 1 check, the latest on 28 September 2026: the link answers
- 28 September 2026: the link answers
6 files
- TBI Microbiome.ipynb, Jupyter, 1,633 lines, 4 matches
- TBI_CLR.Rmd, R, 324 lines, 2 matches
- TBI_Microbiome.ipynb, Jupyter, 1,577 lines, 4 matches
- TBI_Microbiome_R.Rmd, R, 480 lines, 2 matches
- perform_PCoA.py, Python, 139 lines
- README.md, Text, 4 lines
The paper's code and data availability statement is in the Data section.
Tracing map
Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.
What the map holds:
- 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
- 5 scripts, each with its path and the digest of its content;
- 12 matches between paragraphs of the paper and lines of the code (method lexical-v1);
- neither the text of the paper nor the code itself.
Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.
Data
Data links
- ncbi.nlm.nih.gov/
bioproject/ , NCBI; found in “Data Availability”1111907
Data Availability
Raw 16S rRNA sequences generated in this study are available via the NCBI BioProject database under accession number PRJNA1111907 (http://
Reproduced under the paper's license (CC BY), from the paper cited above.
Versions
The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.
Version 1, 28 September 2026: the first record
Recorded: type, language, journal, volume, issue, pages, dates, 4 authors, 8 MeSH terms, 1 funder, 68 references.
Cite
This paper
Pelland, Z. J., Zafar, A., Ay, A. A., & Belanger, K. D. (2026). Non-concussive head impacts sustained during American football correlate with changes in gut microbiome diversity and composition. PloS one, 21(5), e0345651. https://
BibTeX
@article{pelland2026non,
author = {Pelland, Zachary J. and Zafar, Aziz and Ay, Ahmet A. and Belanger, Kenneth Douglas},
title = {{Non-concussive head impacts sustained during American football correlate with changes in gut microbiome diversity and composition}},
journal = {PloS one},
year = {2026},
month = may,
volume = {21},
number = {5},
pages = {e0345651},
publisher = {PLOS},
issn = {1932-6203},
doi = {10.1371/
url = {https://
pmid = {42090386},
pmcid = {PMC13148679}
}
RIS
TY - JOUR
AU - Pelland, Zachary J.
AU - Zafar, Aziz
AU - Ay, Ahmet A.
AU - Belanger, Kenneth Douglas
TI - Non-concussive head impacts sustained during American football correlate with changes in gut microbiome diversity and composition
T2 - PloS one
J2 - PLoS One
PY - 2026
DA - 2026/
VL - 21
IS - 5
SP - e0345651
SN - 1932-6203
PB - PLOS
DO - 10.1371/
UR - https://
LA - en
ER -
CSL-JSON
{
"id": "10.1371/
"type": "article-journal",
"title": "Non-concussive head impacts sustained during American football correlate with changes in gut microbiome diversity and composition",
"container-title": "PloS one",
"author": [
{
"family": "Pelland",
"given": "Zachary J."
},
{
"family": "Zafar",
"given": "Aziz"
},
{
"family": "Ay",
"given": "Ahmet A."
},
{
"family": "Belanger",
"given": "Kenneth Douglas"
}
],
"container-title-short":
"volume": "21",
"issue": "5",
"page": "e0345651",
"DOI": "10.1371/
"PMID": "42090386",
"PMCID": "PMC13148679",
"ISSN": "1932-6203",
"publisher": "PLOS",
"URL": "https://
"language": "en",
"issued": {
"date-parts": [
[
2026,
5,
6
]
]
}
}
The tracing map gets a citation of its own once an author has validated it and it has a DOI.
Similar papers
The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.
- [1] doi:10.1093/braincomms/fcag176 [code]
- Tau topography subtypes account for clinical heterogeneity and longitudinal trajectories in early-onset Alzheimer's disease.Journal: Brain communicationsIn common: car, lmerTest, lme4, 9 other tools, clinical / translational
- [2] doi:10.34133/csbj.0042 [code]
- Using Steady-State Visual Evoked Potentials to Characterize Wide-Ranging Retinopathy Linked to &
lt;i& gt;CRB1& lt;/ i& gt;: Implications for Clinical Trials. Journal: Computational and structural biotechnology journalIn common: car, lmerTest, lme4, 9 other tools, clinical / translational - [3] doi:10.1016/j.xcrm.2026.102766 [code]
- A longitudinal single-cell and spatial multiomic atlas of pediatric high-grade glioma.Journal: Cell reports. MedicineIn common: car, igraph, lmerTest, 9 other tools
- [4] doi:10.1016/j.nicl.2026.104012 [code]
- Structural-functional multilayer brain network properties and outcome of combined repetitive transcranial magnetic stimulation and psychotherapy for obsessive-compulsive disorder.Journal: NeuroImage. ClinicalIn common: car, lmerTest, lme4, 9 other tools
- [5] doi:10.1162/imag.a.105 [code]
- Right posterior theta reflects human parahippocampal phase resetting by salient cues during goal-directed navigationJournal: n/aIn common: car, lmerTest, lme4, 8 other tools
- [6] doi:10.3389/fnhum.2026.1839961 [code]
- Developmental stability of task-rest neural efficiency in youth using a threat and cognitive control task.Journal: Frontiers in human neuroscienceIn common: car, lmerTest, lme4, 8 other tools
- [7] doi:10.1093/jbmrpl/ziag077 [code]
- Sex- and site-specific reference data for size-invariant properties using multi-stack HRpQCT.Journal: JBMR plusIn common: car, lmerTest, lme4, 8 other tools
- [8] doi:10.64898/2026.03.09.710596 [code]
- Infant gut microbiomes contribute to metabolic states that impact brain functionJournal: bioRxiv (preprint)In common: car, lmerTest, lme4, 8 other tools
- [9] doi:10.1126/sciadv.adz6517 [code]
- Corticosterone-linked microglial activity underpins sexually dimorphic neuroplasticity after ketamine anesthesia.Journal: Science advancesIn common: scikit-posthocs, lme4, statsmodels, 8 other tools
- [10] doi:10.1093/bioinformatics/btag592 [code]
- Network-based stratification of allele-specific expression reveals patient subgroups in Huntington's disease.Journal: Bioinformatics (Oxford, England)In common: car, igraph, lme4, 8 other tools
Contribute
The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.
Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.
Claim this paper
Correct its record
Say what each link of this record is, remove the ones that are not the paper's, add the ones that are missing. The correction becomes a new version of the record, in its Versions section.
Validate its tracing map
You validate the map as this page shows it: 1 repository of the authors' code, each at its verified commit and with its license, 5 scripts, and 12 matches between paragraphs and code (see the Code and Map sections). It then receives a DOI on Zenodo, with you (your ORCID iD) and OSCR as its creators; the code itself is not deposited.
The map's fingerprint: sha256:f8ad708060df26de…
Add the badge to its README
The badge links the code to this page. Copy one of these into the README of the paper's code: only you decide where it goes, and nothing is changed for you.
Markdown
[, paste the snippet at the top, then “Commit changes…” and, to review it first, “Create a new branch and start a pull request”. You open the pull request; OSCR asks for no permission.
Request its removal
To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).
Discussion, reproductions, activity
Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.
Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.
Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.
