{ "cells": [ { "cell_type": "markdown", "id": "6f6aa857", "metadata": {}, "source": [ "# Exercise window functions: compute the cumulative number of cases across time, per diet group\n", "\n", "The variable `toevent` contains the time that patients where followed up. We want to calculate the number of events as a function of the follow-up time, separatedely for each diet group. We expect that, if the mediterranean diet has an effect, then over time there will be more cases appearing on the control group in comparison to the other diet groups. \n", "\n", "Here is how to proceed:\n", "- Use a window function to compute the cumulative number of events for each diet group separatedly. As we are interested in the follow-up time, you need to sort the events by the follow-up time first (`toevent`), and then calculate the cumulative sum of events, separatedely per group.\n", "- Add the result as a new column called `'cumulative_event_count'`\n", "\n", "With your new awesome vectorization skills, these two steps should take only one line!\n", "\n", "When ready, execute the code at the end, which has already code that creates a visualiation with the cumulative number of events per group, as a function of the time of follow-up." ] }, { "cell_type": "code", "execution_count": 1, "id": "8f9bc8b1", "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "import seaborn as sns\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "id": "1be11d54", "metadata": {}, "source": [ "### Load patient data" ] }, { "cell_type": "code", "execution_count": 2, "id": "8dfc3020", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
patient-idlocation-idsexagesmokebmiwaistwthhtndiabhypercholfamhisthormop14toeventeventgroupCity
011Female77Never25.92940.657343YesNoYesYesNo95.5386720MedDiet + VOOMadrid
121Female68Never34.851500.949367YesNoYesYesNaN103.0636550MedDiet + NutsMadrid
231Female66Never37.501200.750000YesYesNoNoNo65.5906910MedDiet + NutsMadrid
341Female77Never29.26930.628378YesYesNoNoNo65.4565370MedDiet + VOOMadrid
451Female60Never30.021040.662420YesNoYesNoNo92.7460640ControlMadrid
.........................................................
624012535Male79Never25.281050.640244YesNoYesNoNo85.8288840MedDiet + VOOMalaga
624112545Male62Former27.101040.594286YesNoYesYesNo95.0677620MedDiet + NutsMalaga
624212555Female65Never35.021030.686667YesNoYesNoNo101.9931550MedDiet + VOOMalaga
624312565Male61Never28.42940.576687YesYesNoNoNo92.0396990MedDiet + NutsMalaga
624412575Male58Former24.43930.547059YesYesYesNoNo92.5900070MedDiet + NutsMalaga
\n", "

6245 rows × 18 columns

\n", "
" ], "text/plain": [ " patient-id location-id sex age smoke bmi waist wth \\\n", "0 1 1 Female 77 Never 25.92 94 0.657343 \n", "1 2 1 Female 68 Never 34.85 150 0.949367 \n", "2 3 1 Female 66 Never 37.50 120 0.750000 \n", "3 4 1 Female 77 Never 29.26 93 0.628378 \n", "4 5 1 Female 60 Never 30.02 104 0.662420 \n", "... ... ... ... ... ... ... ... ... \n", "6240 1253 5 Male 79 Never 25.28 105 0.640244 \n", "6241 1254 5 Male 62 Former 27.10 104 0.594286 \n", "6242 1255 5 Female 65 Never 35.02 103 0.686667 \n", "6243 1256 5 Male 61 Never 28.42 94 0.576687 \n", "6244 1257 5 Male 58 Former 24.43 93 0.547059 \n", "\n", " htn diab hyperchol famhist hormo p14 toevent event group \\\n", "0 Yes No Yes Yes No 9 5.538672 0 MedDiet + VOO \n", "1 Yes No Yes Yes NaN 10 3.063655 0 MedDiet + Nuts \n", "2 Yes Yes No No No 6 5.590691 0 MedDiet + Nuts \n", "3 Yes Yes No No No 6 5.456537 0 MedDiet + VOO \n", "4 Yes No Yes No No 9 2.746064 0 Control \n", "... ... ... ... ... ... ... ... ... ... \n", "6240 Yes No Yes No No 8 5.828884 0 MedDiet + VOO \n", "6241 Yes No Yes Yes No 9 5.067762 0 MedDiet + Nuts \n", "6242 Yes No Yes No No 10 1.993155 0 MedDiet + VOO \n", "6243 Yes Yes No No No 9 2.039699 0 MedDiet + Nuts \n", "6244 Yes Yes Yes No No 9 2.590007 0 MedDiet + Nuts \n", "\n", " City \n", "0 Madrid \n", "1 Madrid \n", "2 Madrid \n", "3 Madrid \n", "4 Madrid \n", "... ... \n", "6240 Malaga \n", "6241 Malaga \n", "6242 Malaga \n", "6243 Malaga \n", "6244 Malaga \n", "\n", "[6245 rows x 18 columns]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = pd.read_csv('processed_data_predimed.csv')\n", "df['event'] = df['event'].map({'Yes': 1, 'No': 0})\n", "df" ] }, { "cell_type": "code", "execution_count": 3, "id": "db0d21f7-033e-48ca-9c90-81451af57003", "metadata": {}, "outputs": [], "source": [ "# calculate cumulative number of cases across time, independently for each group\n", "\n", "# your code here:\n", "\n", "\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "id": "7ee10840-12ce-491b-b48a-5fb04df22919", "metadata": {}, "source": [ "If you do it right, the following code will create a visualization as shown in the slides. Uncomment it" ] }, { "cell_type": "code", "execution_count": 4, "id": "de6ee1a8", "metadata": {}, "outputs": [], "source": [ "#sns.lineplot(data=df.sort_values('toevent'), x='toevent', y='cumulative_event_count', hue='group')\n", "#plt.ylabel('Cumulative events')\n", "#plt.xlabel('Years of follow up (variable `toevent`)')\n", "#sns.despine()" ] }, { "cell_type": "markdown", "id": "b8f7a0b7-1ae0-470c-a153-59dc8a6caa28", "metadata": {}, "source": [ "### Optional exercise\n", "\n", "Redo the plot but with the cummulative *percentage* of cases. For that you need to divide the cummulative count by the total number of cases in each group. " ] }, { "cell_type": "code", "execution_count": 5, "id": "d63a56b1", "metadata": {}, "outputs": [], "source": [ "# your code here:\n" ] }, { "cell_type": "code", "execution_count": null, "id": "ffd0e784-ab11-4073-a14a-038ef87c5464", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.5" } }, "nbformat": 4, "nbformat_minor": 5 }