call_peaks

Call peaks for the given Calling Cards data.

The kwargs parameter is used to pass additional arguments into underlying functions. Currently, the following are configured: - pranges_rename_dict: a dictionary that maps the column names in the promoter data to the column names in the PyRanges object. This is used to rename the columns in the PyRanges object after the promoter data is read in. The default is {“chr”: “Chromosome”, “start”: “Start”, “end”: “End”, “strand”: “Strand”}. - join_validate: the validation method to use when joining the promoter data with the experiment and background data. The default is “one_to_one”. - background_total_hops: the total number of hops in the background data. The default is the number of hops in the background data, calculated from the input background data file - experiment_total_hops: the total number of hops in the experiment data. The default is the number of hops in the experiment data, calculated from the input experiment data file - genomic_only: set this flag to include only genomic chromosomes in the experiment and background. See read_in__data for more details. Passed in kwargs so that if none is passed to add_metrics, the default in enrichment_vectorized() and poisson_pval_vectorized() is used. - pseudocount: pseudocount to use when calculating enrichment and poisson pvalue. See either function for more documentation. Passed through kwargs so that if none is passed to add_metrics, the default in enrichment_vectorized() and poisson_pval_vectorized() is used.

:param experiment_data_paths: path(s) to the hops (experiment) data file(s). If multiple paths are provided, they will be concatenated, according to the deduplicate and genomic_only flags, prior to processing. On the concatenated data, however, the deduplicated flag is set to False, since within each file file the data was deduplicated, if it was set to True, and in the concatenated data, multiple hops at the same location is meaningful. :type experiment_data_paths: list :param experiment_orig_chr_convention: the chromosome naming convention used in the experiment data file. :type experiment_orig_chr_convention: str :param promoter_data_path: path to the promoter data file. :type promoter_data_path: str :param promoter_orig_chr_convention: the chromosome naming convention used in the promoter data file. :type promoter_orig_chr_convention: str :param background_data_path: path to the background data file. :type background_data_path: str :param background_orig_chr_convention: the chromosome naming convention used in the background data file. :type background_orig_chr_convention: str :param chrmap_data_path: path to the chromosome map file. :type chrmap_data_path: str :param deduplicate_experiment: If this is true, the experiment data will be deduplicated based on chr, start and end such that if an insertion is found at the same coordinate on different strands, only one of those records will be retained. see read_in_experiment_data for more details. :type deduplicate_experiment: bool :param unified_chr_convention: the chromosome naming convention to use in the output DataFrame. :type unified_chr_convention: str

:return: a pandas DataFrame of promoter regions with Calling Cards metrics. :rtype: DataFrame

Source code in callingcardstools/PeakCalling/yeast/call_peaks.py

def call_peaks(
    experiment_data_paths: list,
    experiment_orig_chr_convention: str,
    promoter_data_path: str,
    promoter_orig_chr_convention: str,
    background_data_path: str,
    background_orig_chr_convention: str,
    chrmap_data_path: str,
    unified_chr_convention: str = "ucsc",
    deduplicate_experiment: bool = True,
    **kwargs,
) -> pd.DataFrame:
    """
    Call peaks for the given Calling Cards data.

    The kwargs parameter is used to pass additional arguments into underlying
    functions. Currently, the following are configured:
    - pranges_rename_dict: a dictionary that maps the column names in the
        promoter data to the column names in the PyRanges object. This is used
        to rename the columns in the PyRanges object after the promoter data
        is read in. The default is {"chr": "Chromosome", "start": "Start",
        "end": "End", "strand": "Strand"}.
    - join_validate: the validation method to use when joining the promoter
        data with the experiment and background data. The default is
        "one_to_one".
    - background_total_hops: the total number of hops in the background data.
        The default is the number of hops in the background data, calculated from
        the input background data file
    - experiment_total_hops: the total number of hops in the experiment data.
        The default is the number of hops in the experiment data, calculated from
        the input experiment data file
    - genomic_only: set this flag to include only genomic chromosomes in the
        experiment and background. See read_in_<experiment/background>_data for
        more details. Passed in kwargs so that if none is passed to add_metrics,
        the default in enrichment_vectorized() and poisson_pval_vectorized() is used.
    - pseudocount: pseudocount to use when calculating enrichment and poisson
        pvalue. See either function for more documentation. Passed through
        kwargs so that if none is passed to add_metrics, the default in
        enrichment_vectorized() and poisson_pval_vectorized() is used.

    :param experiment_data_paths: path(s) to the hops (experiment) data file(s). If
        multiple paths are provided, they will be concatenated, according to the
        `deduplicate` and `genomic_only` flags, prior to processing. On the
        concatenated data, however, the `deduplicated` flag is set to `False`, since
        within each file file the data was deduplicated, if it was set to `True`, and
        in the concatenated data, multiple hops at the same location is meaningful.
    :type experiment_data_paths: list
    :param experiment_orig_chr_convention: the chromosome naming convention
        used in the experiment data file.
    :type experiment_orig_chr_convention: str
    :param promoter_data_path: path to the promoter data file.
    :type promoter_data_path: str
    :param promoter_orig_chr_convention: the chromosome naming convention
        used in the promoter data file.
    :type promoter_orig_chr_convention: str
    :param background_data_path: path to the background data file.
    :type background_data_path: str
    :param background_orig_chr_convention: the chromosome naming convention
        used in the background data file.
    :type background_orig_chr_convention: str
    :param chrmap_data_path: path to the chromosome map file.
    :type chrmap_data_path: str
    :param deduplicate_experiment: If this is true, the experiment data will be
        deduplicated based on `chr`, `start` and `end` such that if an insertion
        is found at the same coordinate on different strands, only one of those records
        will be retained. see `read_in_experiment_data` for more details.
    :type deduplicate_experiment: bool
    :param unified_chr_convention: the chromosome naming convention
        to use in the output DataFrame.
    :type unified_chr_convention: str

    :return: a pandas DataFrame of promoter regions with Calling Cards
        metrics.
    :rtype: DataFrame
    """
    if not isinstance(experiment_data_paths, list):
        raise ValueError(
            "experiment_data_paths must be a list of paths to the experiment data files."
        )
    if len(experiment_data_paths) > 1:
        logger.info(
            "Multiple experiment data files provided. These will be concatenated. "
            "The concatenated data will not be deduplicated, but each file within "
            "the concatenated data will be deduplicated if the `deduplicate` "
            "flag is set to `True`."
        )
    # Read in the chr map
    chrmap_df = read_in_chrmap(
        chrmap_data_path,
        {
            experiment_orig_chr_convention,
            promoter_orig_chr_convention,
            background_orig_chr_convention,
            unified_chr_convention,
        },
    )

    # Read in the promoter and background data
    promoter_df = read_in_promoter_data(
        promoter_data_path,
        promoter_orig_chr_convention,
        unified_chr_convention,
        chrmap_df,
    )

    read_in_data_kwargs = {}
    if "genomic_only" in kwargs:
        read_in_data_kwargs["genomic_only"] = kwargs["genomic_only"]

    # Initialize containers for experiment data
    all_experiment_pr = []
    all_experiment_total_hops = 0

    # Process each experiment data file
    for experiment_data_path in experiment_data_paths:
        experiment_pr, experiment_total_hops = read_in_experiment_data(
            experiment_data_path,
            experiment_orig_chr_convention,
            unified_chr_convention,
            chrmap_df,
            deduplicate_experiment,
            **read_in_data_kwargs,
        )
        all_experiment_pr.append(experiment_pr)
        all_experiment_total_hops += experiment_total_hops

    # Concatenate all experiment data
    concatenated_experiment_pr = pr.concat(all_experiment_pr)

    # Read and process the background data
    background_pr, background_total_hops = read_in_background_data(
        background_data_path,
        background_orig_chr_convention,
        unified_chr_convention,
        chrmap_df,
        **read_in_data_kwargs,
    )

    pyranges_rename_dict = kwargs.get(
        "pranges_rename_dict",
        {"chr": "Chromosome", "start": "Start", "end": "End", "strand": "Strand"},
    )

    promoters_pr = promoter_pyranges(promoter_df, pyranges_rename_dict)

    experiment_hops_df = count_hops(
        promoters_pr, concatenated_experiment_pr, "experiment_hops"
    ).set_index("name", drop=True)

    background_hops_df = count_hops(
        promoters_pr, background_pr, "background_hops"
    ).set_index("name", drop=True)

    promoter_hops_df = (
        promoter_df.drop("score", axis=1)
        .set_index("name")
        .join(
            [experiment_hops_df, background_hops_df],
            how="left",
            validate=kwargs.get("join_validate", "one_to_one"),
        )
        .fillna(0)
        .assign(
            background_total_hops=kwargs.get(
                "background_total_hops", background_total_hops
            ),
            experiment_total_hops=kwargs.get(
                "experiment_total_hops", all_experiment_total_hops
            ),
        )
        .astype(
            {
                "background_hops": "int64",
                "experiment_hops": "int64",
                "background_total_hops": "int64",
                "experiment_total_hops": "int64",
            }
        )
        .reset_index()
    )

    # Extract the add_metric kwargs if they are present
    add_metric_kwargs = {}
    if "pseudocount" in kwargs:
        add_metric_kwargs["pseudocount"] = kwargs["pseudocount"]

    start_time = time.time()
    result_df = add_metrics(promoter_hops_df, **add_metric_kwargs)
    logger.debug(
        "Time taken to process %s promoters: %s seconds",
        len(promoter_hops_df),
        time.time() - start_time,
    )

    return result_df