我从一个CSV格式的API中获取数据，我将其读入一个pandas dataframe，如下所示：

import io
import requests
import pandas as pd
response = requests.get(url, params=params, headers=headers)
df = pd.read_csv(io.StringIO(response.content.decode("utf-8")))

(As在注解中请求，这里是response.content.decode("utf-8")的前2行）

dataset_version,dataset_generated_datetime,dataset_linked_data_default,last_updated_datetime,xml_lang,default_currency,humanitarian,hierarchy,linked_data_uri,budget_not_provided,iati_identifier,reporting_org_ref,reporting_org_type,reporting_org_secondary_reporter,reporting_org_narrative,reporting_org_narrative_xml_lang,title_narrative,title_narrative_xml_lang,description_type,description_narrative,description_narrative_xml_lang,participating_org_ref,participating_org_type,participating_org_role,participating_org_activity_id,participating_org_crs_channel_code,participating_org_narrative,participating_org_narrative_xml_lang,other_identifier_ref,other_identifier_type,other_identifier_owner_org_ref,other_identifier_owner_org_narrative,other_identifier_owner_org_narrative_xml_lang,activity_status_code,activity_date_type,activity_date_iso_date,activity_date_narrative,activity_date_narrative_xml_lang,contact_info_type,contact_info_organisation_narrative,contact_info_organisation_narrative_xml_lang,contact_info_department_narrative,contact_info_department_narrative_xml_lang,contact_info_person_name_narrative,contact_info_person_name_narrative_xml_lang,contact_info_job_title_narrative,contact_info_job_title_narrative_xml_lang,contact_info_telephone,contact_info_email,contact_info_website,contact_info_mailing_address_narrative,contact_info_mailing_address_narrative_xml_lang,activity_scope_code,recipient_country_code,recipient_country_percentage,recipient_country_narrative,recipient_country_narrative_xml_lang,recipient_region_code,recipient_region_vocabulary,recipient_region_vocabulary_uri,recipient_region_percentage,recipient_region_narrative,recipient_region_narrative_xml_lang,location_ref,location_location_reach_code,location_location_id_code,location_location_id_vocabulary,location_name_narrative,location_name_narrative_xml_lang,location_description_narrative,location_description_narrative_xml_lang,location_activity_description_narrative,location_activity_description_narrative_xml_lang,location_administrative_code,location_administrative_vocabulary,location_administrative_level,location_point_srsName,location_point_pos,location_exactness_code,location_location_class_code,location_feature_designation_code,sector_vocabulary,sector_vocabulary_uri,sector_code,sector_percentage,sector_narrative,sector_narrative_xml_lang,tag_code,tag_vocabulary,tag_vocabulary_uri,tag_narrative,tag_narrative_xml_lang,country_budget_items_vocabulary,country_budget_items_budget_item_code,country_budget_items_budget_item_percentage,country_budget_items_budget_item_description_narrative,country_budget_items_budget_item_description_narrative_xml_lang,humanitarian_scope_type,humanitarian_scope_vocabulary,humanitarian_scope_vocabulary_uri,humanitarian_scope_code,humanitarian_scope_narrative,humanitarian_scope_narrative_xml_lang,policy_marker_vocabulary,policy_marker_vocabulary_uri,policy_marker_code,policy_marker_significance,policy_marker_narrative,policy_marker_narrative_xml_lang,collaboration_type_code,default_flow_type_code,default_finance_type_code,default_aid_type_code,default_aid_type_vocabulary,default_tied_status_code,budget_type,budget_status,budget_period_start_iso_date,budget_period_end_iso_date,budget_value,budget_value_currency,budget_value_value_date,planned_disbursement_type,planned_disbursement_period_start_iso_date,planned_disbursement_period_end_iso_date,planned_disbursement_value,planned_disbursement_value_currency,planned_disbursement_value_value_date,planned_disbursement_provider_org_ref,planned_disbursement_provider_org_provider_activity_id,planned_disbursement_provider_org_type,planned_disbursement_provider_org_narrative,planned_disbursement_provider_org_narrative_xml_lang,planned_disbursement_receiver_org_ref,planned_disbursement_receiver_org_receiver_activity_id,planned_disbursement_receiver_org_type,planned_disbursement_receiver_org_narrative,planned_disbursement_receiver_org_narrative_xml_lang,capital_spend_percentage,transaction_ref,transaction_humanitarian,transaction_transaction_type_code,transaction_transaction_date_iso_date,transaction_value,transaction_value_currency,transaction_value_value_date,transaction_description_narrative,transaction_description_narrative_xml_lang,transaction_provider_org_ref,transaction_provider_org_provider_activity_id,transaction_provider_org_type,transaction_provider_org_narrative,transaction_provider_org_narrative_xml_lang,transaction_receiver_org_ref,transaction_receiver_org_receiver_activity_id,transaction_receiver_org_type,transaction_receiver_org_narrative,transaction_receiver_org_narrative_xml_lang,transaction_disbursement_channel_code,transaction_sector_vocabulary,transaction_sector_vocabulary_uri,transaction_sector_code,transaction_sector_narrative,transaction_sector_narrative_xml_lang,transaction_recipient_country_code,transaction_recipient_country_narrative,transaction_recipient_country_narrative_xml_lang,transaction_recipient_region_code,transaction_recipient_region_vocabulary,transaction_recipient_region_vocabulary_uri,transaction_recipient_region_narrative,transaction_recipient_region_narrative_xml_lang,transaction_flow_type_code,transaction_finance_type_code,transaction_aid_type_code,transaction_aid_type_vocabulary,transaction_tied_status_code,document_link_url,document_link_format,document_link_title_narrative,document_link_title_narrative_xml_lang,document_link_description_narrative,document_link_description_narrative_xml_lang,document_link_category_code,document_link_language_code,document_link_document_date_iso_date,related_activity_ref,related_activity_type,legacy_data_name,legacy_data_value,legacy_data_iati_equivalent,conditions_attached,conditions_condition_type,conditions_condition_narrative,conditions_condition_narrative_xml_lang,result_type,result_aggregation_status,result_title_narrative,result_title_narrative_xml_lang,result_description_narrative,result_description_narrative_xml_lang,result_document_link_url,result_document_link_format,result_document_link_title_narrative,result_document_link_title_narrative_xml_lang,result_document_link_description_narrative,result_document_link_description_narrative_xml_lang,result_document_link_category_code,result_document_link_language_code,result_document_link_document_date_iso_date,result_reference_code,result_reference_vocabulary,result_reference_vocabulary_uri,result_indicator_measure,result_indicator_ascending,result_indicator_aggregation_status,result_indicator_title_narrative,result_indicator_title_narrative_xml_lang,result_indicator_description_narrative,result_indicator_description_narrative_xml_lang,result_indicator_document_link_url,result_indicator_document_link_format,result_indicator_document_link_title_narrative,result_indicator_document_link_title_narrative_xml_lang,result_indicator_document_link_description_narrative,result_indicator_document_link_description_narrative_xml_lang,result_indicator_document_link_category_code,result_indicator_document_link_language_code,result_indicator_document_link_document_date_iso_date,result_indicator_reference_vocabulary,result_indicator_reference_code,result_indicator_reference_indicator_uri,result_indicator_baseline_iso_date,result_indicator_baseline_year,result_indicator_baseline_value,result_indicator_baseline_location_ref,result_indicator_baseline_dimension_name,result_indicator_baseline_dimension_value,result_indicator_baseline_document_link_url,result_indicator_baseline_document_link_format,result_indicator_baseline_document_link_title_narrative,result_indicator_baseline_document_link_title_narrative_xml_lang,result_indicator_baseline_document_link_description_narrative,result_indicator_baseline_document_link_description_narrative_xml_lang,result_indicator_baseline_document_link_category_code,result_indicator_baseline_document_link_language_code,result_indicator_baseline_document_link_document_date_iso_date,result_indicator_baseline_comment_narrative,result_indicator_baseline_comment_narrative_xml_lang,result_indicator_period_period_start_iso_date,result_indicator_period_period_end_iso_date,result_indicator_period_target_value,result_indicator_period_target_location_ref,result_indicator_period_target_dimension_name,result_indicator_period_target_dimension_value,result_indicator_period_target_comment_narrative,result_indicator_period_target_comment_narrative_xml_lang,result_indicator_period_target_document_link_url,result_indicator_period_target_document_link_format,result_indicator_period_target_document_link_title_narrative,result_indicator_period_target_document_link_title_narrative_xml_lang,result_indicator_period_target_document_link_description_narrative,result_indicator_period_target_document_link_description_narrative_xml_lang,result_indicator_period_target_document_link_category_code,result_indicator_period_target_document_link_language_code,result_indicator_period_target_document_link_document_date_iso_date,result_indicator_period_actual_value,result_indicator_period_actual_location_ref,result_indicator_period_actual_dimension_name,result_indicator_period_actual_dimension_value,result_indicator_period_actual_comment_narrative,result_indicator_period_actual_comment_narrative_xml_lang,result_indicator_period_actual_document_link_url,result_indicator_period_actual_document_link_format,result_indicator_period_actual_document_link_title_narrative,result_indicator_period_actual_document_link_title_narrative_xml_lang,result_indicator_period_actual_document_link_description_narrative,result_indicator_period_actual_document_link_description_narrative_xml_lang,result_indicator_period_actual_document_link_category_code,result_indicator_period_actual_document_link_language_code,result_indicator_period_actual_document_link_document_date_iso_date,crs_add_other_flags_code,crs_add_other_flags_significance,crs_add_loan_terms_rate_1,crs_add_loan_terms_rate_2,crs_add_loan_terms_repayment_type_code,crs_add_loan_terms_repayment_plan_code,crs_add_loan_terms_commitment_date_iso_date,crs_add_loan_terms_repayment_first_date_iso_date,crs_add_loan_terms_repayment_final_date_iso_date,crs_add_loan_status_year,crs_add_loan_status_currency,crs_add_loan_status_value_date,crs_add_loan_status_interest_received,crs_add_loan_status_principal_outstanding,crs_add_loan_status_principal_arrears,crs_add_loan_status_interest_arrears,crs_add_channel_code,fss_extraction_date,fss_priority,fss_phaseout_year,fss_forecast,fss_forecast_year,fss_forecast_currency,fss_forecast_value_date
2.03,2023-02-16T09:12:32Z,,2023-02-16T09:12:32Z,EN,USD,,1,,,XI-IATI-WBTF-TF072664,XI-IATI-WBTF,40,,World Bank Trust Funds,,Saint Lucia: Disaster Vulnerability Reduction Project EDF Single Donor Trust Fund,,1,This EDF Trust Fund financed by the European Commission aims to provide additional financing to the original Saint Lucia Disaster Vulnerability Reduction Project,,"TF612001,XI-IATI-WBTF","40,40","1,4",,,"EU-Commission of the European Communities,World Bank Trust Funds",,,,,,,2,"1,2,3","2015-09-01T00:00:00Z,2015-09-01T00:00:00Z,2023-12-31T00:00:00Z",,,3,,,"HIS-Urban\, Rural & Soc Dev - GP",,,,,,1202-458-7916,,,"1818 H St\, NW\, Washington DC - 22036",,4,LC,,SAINT LUCIA,,,,,,,,,,,,,,,,,,,,,,,,,,"1,1,1,1",,"11120,12230,14021,74020","11.0,11.0,4.0,74.0","+![cdata[Education facilities and training]]+,+![cdata[Basic health infrastructure]]+,+![cdata[Water supply - large systems]]+,+![cdata[Multi-hazard response preparedness]]+",,,,,,,,,,,,,,,,,,,,,,,,2,,,B03,,5,1,,2015-09-01T00:00:00Z,2023-12-31T00:00:00Z,6935451.25,USD,2023-01-31T00:00:00Z,,,,,,,,,,,,,,,,,,,,"1,1,11,2,2,2,2","2020-06-30T00:00:00Z,2022-08-31T00:00:00Z,2020-06-30T00:00:00Z,2020-06-30T00:00:00Z,2020-06-30T00:00:00Z,2022-09-30T00:00:00Z,2022-10-31T00:00:00Z","5519956.0,1236173.25,6935451.25,5341367.39,73469.52,87015.94,1098401.07","USD,USD,USD,USD,USD,USD,USD","2023-01-31T00:00:00Z,2023-01-31T00:00:00Z,2023-01-31T00:00:00Z,2023-01-31T00:00:00Z,2023-01-31T00:00:00Z,2023-01-31T00:00:00Z,2023-01-31T00:00:00Z","Incoming Funds (Cumulative Paid-In Contribution )-as of June 30th\, 2020,Incoming Funds (Paid-In Contribution ) -  for August \, 2022,Incoming Commitment(Cumulative Signed Contribution)-as of June 30th\, 2020,Outgoing Commitment ( Total Grant Allocated Amount to Project ) - as of June 30th\, 2020,Allocation to Bank executed activities - as of June 30th\, 2020,Allocation to Bank executed activities -  for September \, 2022,Outgoing Commitment ( Grant Allocated Amount to Project ) -  for October \, 2022",,,"TF612001,TF612001,TF612001,XI-IATI-WBTF-TF072664,XI-IATI-WBTF-TF072664,XI-IATI-WBTF-TF072664,XI-IATI-WBTF-TF072664",,"EU-Commission of the European Communities,EU-Commission of the European Communities,EU-Commission of the European Communities,Saint Lucia: Disaster Vulnerability Reduction Project EDF Single Donor Trust Fund,Saint Lucia: Disaster Vulnerability Reduction Project EDF Single Donor Trust Fund,Saint Lucia: Disaster Vulnerability Reduction Project EDF Single Donor Trust Fund,Saint Lucia: Disaster Vulnerability Reduction Project EDF Single Donor Trust Fund",,,"XI-IATI-WBTF-TF072664,XI-IATI-WBTF-TF072664,XI-IATI-WBTF-TF072664,XI-IATI-WBTF-P127226,XI-IATI-WBTF-P127226,XI-IATI-WBTF-P127226,XI-IATI-WBTF-P127226",,"Saint Lucia: Disaster Vulnerability Reduction Project EDF Single Donor Trust Fund,Saint Lucia: Disaster Vulnerability Reduction Project EDF Single Donor Trust Fund,Saint Lucia: Disaster Vulnerability Reduction Project EDF Single Donor Trust Fund,SAINT LUCIA,SAINT LUCIA,SAINT LUCIA,SAINT LUCIA",,,,,,,,,,,,,,,,,"110,110,110,110,110,110,110",,,,,,,,,,,,,XI-IATI-WBTF-P127226,4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

经检查， Dataframe 中的许多单元格包含长度不同的逗号分隔数据。逗号分隔值未用引号括起来，并且某些值包含使用反斜杠转义的逗号。示例列如下所示：

eg = pd.DataFrame({"a": ["A single value", "Three,separate,values", "This value contains a \,, and so does\, this one", pd.NA], "b": ["", pd.NA, "2,3,4","12,17"]})

我需要将这些单元格分隔成每个单元格内的列表或单独的列。我可以一次一个单元格地执行此操作，如下所示：

import csv
def to_list_of_strings(cell):
    if pd.isnull(cell):
        return []
    elif "," in cell:
        return next(csv.reader(io.StringIO(cell), escapechar="\\"))
    else:
        return [cell]

result = eg.a.apply(to_list_of_strings)

（我使用csv是希望它能处理可能需要的其他解析，比如引号。）
所需的结果如下所示（对于list(result)）

[['A single value'],
 ['Three', 'separate', 'values'],
 ['This value contains a ,', ' and so does, this one'],
 []]

这在多列和数千行中可能会很慢。有没有更快的矢量化方法来做到这一点？