CHAIMELEON Open Challenges Banner
Join

Classification phase

The Classification Phase will allow participants to propose solutions for both prostate and lung cancer. This phase has been design with the purpose of selecting the top 40 participants that will compete in the Championship phase.

The Classification Phase is based on the following schema:


AI solutions

  1. Prostate cancer

This task aims to differentiate between low-grade and high-grade prostate cancer. The goal is to avoid unnecessary aggressive procedures in low-grade patients while providing the most advanced therapy for high-grade cases.

Therefore, this task is based on a binary classification problem.

  1. Lung cancer

In this task, the objective is to assess baseline CT imaging and radiomics features in a diverse range of non-small cell lung cancer (NSCLC) patients, with the aim of predicting overall survival (OS).

Data access

Once registered and accepted on the challenge, participants will receive by email a single access link to download the training data. The structure of the training data will be the following:

For each patient, the following information will be provided:

  • case_xxxx.nii.gz → Image in compressed NIfTI format. The shared image is the one obtained after a harmonization process applied to the raw data. This harmonization pre-processing is the result of the algorithms developed in the CHAIMELEON project. Depending on the cancer type the raw data is different:

    • Prostate cancer: T2w MRI → Harmonization to guarantee unification in the image contrast across scans.

    • Lung cancer: CT scan → Harmonization to guarantee unification in the image resolution across scans.

  • case_xxxx.json → JSON with selected clinical information of the patient. Depending on the cancer type the clinical variables provided are different (in brackets the key of the variable in the JSON file is specified):

    • Prostate cancer:

      • Patient age (age)

      • Total prostate specific antigen level (PSA) at diagnosis (psa)

    • Lung cancer:

      • Input variables:

        • Gender (gender)

        • Age (age)

        • Smoking status (smoking_status)

  • case_xxxx_ground_truth.json → JSON with the ground truth. Its content will depend on the cancer type:

    • Prostate cancer: String with the risk of the patient. It can have two values “Low” or “High”.

    • Lung cancer: Dictionary with the following information:

      • Survival time (survival_time) → time from diagnosis to death or last follow-up

      • Event (event) → if the patient suffered the event (death)

Evaluation

The evaluation is split into two phases:

  • Validation and tunning: participants can upload the prepared docker image to the “Validation phase” and the corresponding metrics will be extracted using the validation dataset. The associated leaderboard will be updated. The number of applications on the validation dataset will be limited to 3 submissions.

  • Test: To avoid models’ overfitting to the validation dataset, once a final model is selected by the participants, they will upload the docker image to the “Test phase” and the corresponding metrics will be extracted using the test dataset. The associated leaderboard will be updated. A single submission will be allowed in this phase.

Depending on the cancer type the metrics used to update the leaderboard are the following:

  • Prostate cancer: Different metrics will be used to provide a final score. The considered metrics are the following:

    • Area Under the ROC Curve (AUC)

    • Sensitivity

    • Specificity

    • Balanced Accuracy

      The final score will be calculated as: 



  • Lung cancer: Different metrics will be used to provide a final score. The considered metrics are the following:

    • Concordance Index (C-index): measures the model’s ability to correctly rank the predicted OS times. It compares the predicted OS time of one patient to that of another patient and checks if the prediction is consistent with the actual outcome. A higher C-index indicates better performance.

      The final score used to update the leaderboard is based on the C-index.