python 如何在Huggingface上更改数据集格式

inkz8wg9  于 2023-01-08  发布在  Python
关注(0)|答案(1)|浏览(211)

我有一个数据集,看起来像这样:

rowid,url,text,domain_id,domain_id_label,width,height,similarity,punsafe,pwatermark,aesthetic,hash,__index_level_0__
1,https://cdn.idahopotato.com/cache/4075b86c99bc2c46f927f3be5949d161_w310.jpg,Fattoush Salad with Roasted Potatoes,1,cdn.idahopotato.com,310,206,0.32190072536468506,1.9172496e-05,0.042544544,6.0983996,-7769833748550554891,113
2,https://lh3.googleusercontent.com/-Gw5LBM0zYU8/TX46gXdbI2I/AAAAAAAABto/MI-8zA_yR84/s1600/24430-virginia_woolf.jpg,"an analysis of self portrayal in novels by virginia woolf A room of one's own study guide contains a biography of virginia woolf, literature essays, quiz questions, major themes, characters, and a full summary and analysis about a room of one's own a room of one's own summary.",2,lh3.googleusercontent.com,720,1000,0.33762750029563904,1.737107e-06,0.40568027,6.109017,8675719636262469033,877
3,https://www.mediaplaynews.com/wp-content/uploads/2020/11/last-christmas.png,Christmas Comes Early to U.K. Weekly Home Entertainment Chart,3,www.mediaplaynews.com,900,601,0.33475491404533386,0.023501933,0.4089923,6.0230927,-510709293545570516,952
4,https://statesofincarceration.org/sites/default/files/Amy_Tyson-Dorothea_Lynde_Dix_c1850-55.jpg,Amy Garcia Wikipedia a legacy of reform: dorothea dix (1802–1887) | states of,4,statesofincarceration.org,840,1112,0.332019180059433,5.6428403e-06,0.15564087,6.431951,7982521258241828259,1163
5,https://cdn.shopify.com/s/files/1/0094/8653/2687/products/FAM-1202ZT_300x300.jpg?v=1570195462,3D Metal Cornish Harbour Painting,5,cdn.shopify.com,300,300,0.32057473063468933,8.072326e-06,0.10981648,6.1677094,-2541341491343729392,1431
6,http://l.yimg.com/bt/api/res/1.2/DVaATazcS6Jq8DuG8pWqVA--/YXBwaWQ9eW5ld3M7Zmk9ZmlsbDtoPTc1NztweW9mZj0wO3E9NzU7c209MTt3PTk2MA--/http://media.zenfs.com/en_us/News/ap_webfeeds/82c71bbdef270916370f6a7067000065.jpg,"""In this undated photo provided by the New York City Ballet, Robert Fairchild performs in """"In Creases"""" by choreographer Justin Peck which is being performed by the New York City Ballet in New York. (AP Photo/New York City Ballet, Paul Kolnik)""",6,l.yimg.com,960,757,0.3140464425086975,0.00040382147,0.44375956,6.0047417,1037540669530061759,1506
7,https://render.fineartamerica.com/images/rendered/search/flat/round-beach-towel/images/artworkimages/medium/1/winter-wonderland-anne-gifford.jpg?&targetx=0&targety=-159&imagewidth=788&imageheight=1106&modelwidth=788&modelheight=788&backgroundcolor=E5E9ED&orientation=0&removethiseventually=true,Round Beach Towel featuring the painting Winter Wonderland by Anne Gifford,7,render.fineartamerica.com,180,180,0.3693836927413941,1.1998264e-07,0.29982352,6.015644,-2125211670902297681,2101
8,https://en-fmgdesigns.s3.amazonaws.com/53366.jpg?v%5Cx3d9543cd464bebc77aef550c7740b87027%0D%0A,"sun, rays, Spring, Mountains - Beautiful views wallpapers: 2560x1440",8,en-fmgdesigns.s3.amazonaws.com,541,360,0.30625030398368835,7.164156e-06,0.16085383,6.0956593,-8492675886823456363,2397
9,"https://images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com/f/1e3ad9ba-d3bb-4750-9318-aa0d1a7f50b0/d6s1yej-9bf6e835-4d10-4708-a7c1-32143aae41f5.jpg/v1/fill/w_177,h_250,q_70,strp/sailor_moon_by_charlie_bowater_d6s1yej-250t.jpg?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOjdlMGQxODg5ODIyNjQzNzNhNWYwZDQxNWVhMGQyNmUwIiwiaXNzIjoidXJuOmFwcDo3ZTBkMTg4OTgyMjY0MzczYTVmMGQ0MTVlYTBkMjZlMCIsIm9iaiI6W1t7ImhlaWdodCI6Ijw9ODc3IiwicGF0aCI6IlwvZlwvMWUzYWQ5YmEtZDNiYi00NzUwLTkzMTgtYWEwZDFhN2Y1MGIwXC9kNnMxeWVqLTliZjZlODM1LTRkMTAtNDcwOC1hN2MxLTMyMTQzYWFlNDFmNS5qcGciLCJ3aWR0aCI6Ijw9NjIwIn1dXSwiYXVkIjpbInVybjpzZXJ2aWNlOmltYWdlLm9wZXJhdGlvbnMiXX0.I2GILU7wflVd1QXJ0TQRzn4a0jqx_WHnhz1oDWfIR0I",Sailor Moon by Charlie-Bowater,9,images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com,177,250,0.35163620114326477,0.0015848279,0.23917364,6.1056666,7304372338209907934,2400
10,https://konachan.com/data/preview/7c/42/7c42a9164bcfb0fce7c88085f09d9dce.jpg,Rating: Safe Score: 171 Tags: aeolian_(wlop) black_hair braids building butterfly city clouds ghostblade headdress landscape logo long_hair ponytail scenic skirt sky sunset watermark wlop User: luckyluna,10,konachan.com,300,166,0.3197726607322693,0.052469403,0.22856134,6.2950845,2908580222882266365,2704
11,https://render.fineartamerica.com/images/rendered/search/print/images-medium-5/1-lake-in-winter-crater-lake-crater-panoramic-images.jpg,"Lake In Winter, Crater Lake, Crater Art Print",7,render.fineartamerica.com,400,266,0.318795382976532,1.3206559e-06,1.0,6.085382,-8877775580192596443,2988
12,https://dreamlandtours.net/wp-content/uploads/2019/06/IMG_1768-scaled.jpg,Hiker gazes at the Wave with reflection pool,11,dreamlandtours.net,1920,2560,0.2865600287914276,4.9344366e-05,0.10647261,6.072795,-1859498664806039492,3017
13,https://t2.ftcdn.net/jpg/00/53/11/83/400_F_53118382_iVHtfP2XJVZVULdvRulnZlBXctv3mtbK.jpg,beautiful blond woman with flowers of apple tree,12,t2.ftcdn.net,500,334,0.3716212213039398,0.9306521,0.1296275,6.0769506,-1891859739494487481,3148
14,https://i2.wp.com/voxdominus.ru/wp-content/uploads/2021/02/UK_Arpen_6_Seat_Dining_Table-1.jpg?resize=750%2C500&ssl=1,table,13,i2.wp.com,750,500,0.2828073501586914,0.00011419203,0.08425643,6.151825,4236973487256703435,3263
15,http://render.fineartamerica.com/images/rendered/medium/greeting-card/images-medium-5/intense-siamese-cat-painting-print-2-svetlana-novikova.jpg,Giclee Prints Greeting Cards - Intense Siamese Cat painting print 2 Greeting Card by Svetlana Novikova,7,render.fineartamerica.com,646,470,0.3692695200443268,7.591725e-07,0.07571758,6.105506,-7442211688250030372,3353
16,https://thumb9.shutterstock.com/image-photo/stock-photo-bruges-beautiful-scenic-cityscape-with-a-medieval-groenerei-canal-reflections-canal-boat-tour-450w-385227766.jpg,"Bruges, Beautiful Scenic cityscape with a medieval Groenerei canal reflections, canal boat tour embankment jetty and the quay Dijver and Illuminated Church of Our Lady at night in summer, Belgium - stock photo",14,thumb9.shutterstock.com,338,470,0.35601797699928284,0.001455605,0.40309513,6.3248205,2854978087673303320,3546
17,https://s.yimg.com/ny/api/res/1.2/oWf.pIWNf51jf0LXqcZTwA--/YXBwaWQ9aGlnaGxhbmRlcjtzbT0xO3c9MTA1NDtoPTcwMDtpbD1wbGFuZQ--/http://media.zenfs.com/en_US/Sports/AP_General/201405211411510869115-p5.jpg,"President Barack Obama waves to members of the NFL Super Bowl champion Seattle Seahawks football team after welcoming the team to the East Room of the White House in Washington, Wednesday, May 21, 2014. The Seahawks defeated the Denver Broncos in Super Bowl XLVIII. (AP Photo/Susan Walsh)",15,s.yimg.com,1054,700,0.31588438153266907,0.0033192635,0.08139456,6.063474,-5848983857644015815,3611
18,http://render.fineartamerica.com/images/rendered/medium/greeting-card/images-medium-5/family-stroll-phil-jaeger.jpg,African Elephants Greeting Cards - Family Stroll Greeting Card by Phil Jaeger,7,render.fineartamerica.com,455,646,0.3647528290748596,4.121351e-05,0.10197897,6.1357856,-1235949504306333001,3627
19,https://us.123rf.com/450wm/massonforstock/massonforstock1409/massonforstock140900046/31356863-lonely-girl-with-suitcase-at-near-airplane-photo-in-old-image-style.jpg,Lonely girl with suitcase at near airplane. Photo in old image style. photo,16,us.123rf.com,300,450,0.32632535696029663,0.40212536,0.13383819,6.2053504,3222426229479515080,3655
20,https://www.artbrokerage.com/artthumb/roulette_73713_1/360x322/Leon_Roulette_Harborside_At_Portofino_2010_Embellished.jpg,Harborside At Portofino 2010 Embellished Limited Edition Print - Leon Roulette,17,www.artbrokerage.com,360,270,0.3299652934074402,2.903781e-05,0.076089926,6.554896,-7726527588692324976,3675
21,https://gerrymiles.nl/wp-content/uploads/2020/01/Underwater-painting-spadefish-gerry-miles.jpg,Underwater painting of spadefish Spade fishes above a shipwreck by Gerry Miles,18,gerrymiles.nl,1300,1166,0.3595767021179199,2.9819766e-06,0.19193313,6.123009,978872985244231604,3799
22,https://us.123rf.com/450wm/lanabrest/lanabrest1511/lanabrest151100017/47935708-a-fantasy-about-drem-island-with-a-castle-on-it-original-oil-painting-uncharted.jpg?ver=6,cliff: A fantasy about drem island with a castle on it. Original oil painting Uncharted,16,us.123rf.com,450,300,0.3242622911930084,8.0927486e-05,0.22578254,6.2720966,-1996111269057685352,4161
23,https://us.123rf.com/450wm/lightwise/lightwise1512/lightwise151200042/49949750-help-concept-and-corporate-support-as-a-group-of-people-excluded-from-advancing-on-a-drawing-of-a-li.jpg?ver=6,business education: Help concept and corporate support as a group of people excluded from advancing on a drawing of a line that is being erased and sketched by a pencil. Stock Photo,16,us.123rf.com,450,341,0.3243783414363861,0.00043466687,0.6340423,6.0079627,-7188812172422378228,4537
24,https://t2.ftcdn.net/jpg/00/75/71/11/240_F_75711179_golOjcINgc5vbCXyj23AmeaqVcvKp2NM.jpg,close-up portrait of woman in black paint and green leaf on face,12,t2.ftcdn.net,479,240,0.3160702586174011,0.013185352,0.20235766,6.023582,-920255550450707673,4539
25,https://naglly.com/archives/2016/10/04/Mont_Saint_Michel_from_a_drone.png,Mont_Saint_Michel_from_a_drone.png,19,naglly.com,500,229,0.36000365018844604,5.2968724e-05,0.168854,6.2057323,8488573082681257870,4669
26,https://i.imgur.com/ajb54XYm.jpg,burning_planet_by_parad1gm.png,20,i.imgur.com,320,320,0.30053475499153137,0.00073572993,0.004659973,6.1471276,215317020151320997,5501
...

此数据集已在Huggingface上:https://huggingface.co/datasets/oo92/diffusion-dataset/viewer/oo92--diffusion-dataset/train
url列是与text列条目对应的图像的URL。
我正在尝试将此数据集设置为与Pokemon BLIP

相同的格式
我已经把所有的图片下载到了一个单独的文件夹中,但是我不知道如何把数据以这种格式上传到huggingface上。
这就是我尝试使用它的方式:

在这里,而不是口袋妖怪,它的第一个图像和文本从我的数据集。

vi4fp9gy

vi4fp9gy1#

也许我回答得太晚了,但是这里有一个最简单的解决方案,可以获得与数据集pokemon-blip-capcion相同的结果。
对于我的数据集,我创建了元数据.jsonl文件,当我写字符串{“图像”:“2022. jpg”,“文本”:“ neon skull”}。如果愿意,您可以添加更多信息。接下来,您需要将此文件放在包含数据的文件夹中,并使用diffusers库的标准函数加载数据集并推送到hub x1c 0d1xx 1c 1d 1x

相关问题