Answer to Stack Oveflow Question: Select Pandas DF Rows including any string in a list of strings

A possible solution to a Stack Overflow question.

In [42]:
import pandas as pd
In [49]:
# This URL doesn't return CSV.
CSV_URL = "https://drive.google.com/open?id=1rwg8c2GmtqLeGGv1xm9w6kS98iqgd6vW"
# Data file saved from within a browser to help with question.

# I stored the BitcoinData.csv data on my Minio server.
df = pd.read_csv("https://minio.apps.selfip.com/mymedia/csv/BitcoinData.csv")

selected_words = [
    "accept",
    "believe",
    "trust",
    "accepted",
    "accepts",
    "trusts",
    "believes",
    "acceptance",
    "trusted",
    "trusting",
    "accepting",
    "believes",
    "believing",
    "believed",
    "normal",
    "normalize",
    " normalized",
    "routine",
    "belief",
    "faith",
    "confidence",
    "adoption",
    "adopt",
    "adopted",
    "embrace",
    "approve",
    "approval",
    "approved",
    "approves",
]
In [44]:
%%timeit
mask = pd.Series(any(word in item for word in selected_words) for item in df["story"])
18.2 ms ± 115 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
In [45]:
%%timeit

df[mask]
947 µs ± 7.18 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
In [46]:
%%timeit

df[df.story.str.contains('|'.join(selected_words))]
128 ms ± 413 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [50]:
df[mask] == df[df.story.str.contains('|'.join(selected_words))]
Out[50]:
story date sentiment bitcoin price
0 True True True True
1 True True True True
2 True True True True
3 True True True True
5 True True True True
6 True True True True
7 True True True True
8 True True True True
9 True True True True
10 True True True True
13 True True True True
15 True True True True
16 True True True True
17 True True True True
20 True True True True
21 True True True True
22 True True True True
25 True True True True
26 True True True True
28 True True True True
29 True True True True
30 True True True True
31 True True True True
32 True True True True
33 True True True True
34 True True True True
35 True True True True
36 True True True True
37 True True True True
38 True True True True
... ... ... ... ...
372 True True True True
373 True True True True
374 True True True True
375 True True True True
377 True True True True
378 True True True True
379 True True True True
380 True True True True
381 True True True True
382 True True True True
383 True True True True
384 True True True True
386 True True True True
387 True True True True
388 True True True True
390 True True True True
391 True True True True
392 True True True True
393 True True True True
395 True True True True
397 True True True True
398 True True True True
401 True True True True
402 True True True True
404 True True True True
405 True True True True
407 True True True True
408 True True True True
409 True True True True
410 True True True True

336 rows × 4 columns

In [51]:
df
Out[51]:
story date sentiment bitcoin price
0 it was a curious choice for the good wife an e... 1/16/12 0 $6.68
1 when he was a yale law school student reuben g... 4/7/13 0 $162.30
2 video bitcoin has real world investors the tim... 4/11/13 0 $124.90
3 bitcoin s wild ride may not have been the bigg... 4/14/13 0 $90.00
4 amid the incense cheap art and herbal remedies... 5/6/13 1 $112.30
5 san francisco eight years ago ernie allen the ... 5/29/13 0 $132.30
6 12 34 p m updated this post was changed to pro... 5/29/13 1 $132.30
7 agaton strom for the new york timestyler left ... 7/1/13 1 $88.05
8 what is a bitcoin as a virtual currency it is ... 7/1/13 1 $88.05
9 photocredit bitcoin has been promoted as an al... 7/2/13 1 $90.13
10 depending on whom you ask bitcoins are a goofy... 7/31/13 1 $106.09
11 michael appleton for the new york timesbenjami... 8/13/13 0 $109.00
12 photocredit state and federal officials are st... 8/14/13 0 $112.56
13 photobarry silbert the chief executive of seco... 9/25/13 1 $128.22
14 any delay in raising the debt ceiling would h... 9/28/13 0 $134.78
15 nearly everything about silk road was shrouded... 10/2/13 0 $114.13
16 photocredit silkroaddrugs org via associated p... 10/8/13 0 $126.00
17 the currency known as bitcoin a much hyped and... 10/30/13 1 $199.97
18 photoideas with currency chris larsen chief ex... 11/11/13 1 $342.44
19 photobenjamin m lawsky the new york financial ... 11/14/13 0 $420.20
20 photobitcoins have strong appeal for chinese i... 11/17/13 1 $492.11
21 photounlike cash a bitcoin transaction history... 11/18/13 0 $703.56
22 photojennifer shasky calvery left with mythili... 11/18/13 0 $703.56
23 photoa screen shot of fiatleak com which track... 11/19/13 1 $584.61
24 proponents of digital currencies like bitcoins... 11/21/13 1 $722.43
25 this week the chinese government issued its mo... 11/22/13 1 $771.44
26 bitcoin isn t ready for popular consumption an... 11/23/13 0 $797.82
27 two israeli computer scientists say they may h... 11/23/13 0 $797.82
28 video bitcoin believers while regulators debat... 11/24/13 1 $774.25
29 photocredit with mounting interest from promin... 11/25/13 0 $799.11
... ... ... ... ...
381 the russian operatives accused of hacking the ... 7/22/18 1 $7,418.49
382 san francisco hedge funds go to the cayman isl... 7/29/18 1 $8,218.46
383 i m still on vacation hiking and biking in var... 7/31/18 0 $7,780.44
384 san francisco pete roberts of nottingham engla... 8/20/18 0 $6,308.53
385 how do new york times journalists use technolo... 9/12/18 1 $6,351.80
386 massena n y the hulking aluminum plant in this... 9/19/18 0 $6,398.54
387 cryptocurrencies like bitcoin have lost over h... 9/25/18 0 $6,446.47
388 get the dealbook newsletter to make sense of m... 9/25/18 0 $6,446.47
389 get the dealbook newsletter to make sense of m... 9/27/18 1 $6,676.75
390 in november 2010 as the federal reserve embark... 10/1/18 0 $6,589.62
391 storey county nev an enormous plot of land in ... 11/1/18 1 $6,377.78
392 san francisco over the last year charlie shrem... 11/2/18 1 $6,388.44
393 san francisco the news on wall street this wee... 11/21/18 0 $4,602.17
394 two iranians were behind the ransomware attack... 11/28/18 0 $4,257.42
395 there s a large spot reserved in the popular i... 12/11/18 0 $3,424.59
396 get the dealbook newsletter to make sense of m... 12/13/18 0 $3,313.68
397 palo alto calif last year around this time a t... 12/27/18 1 $3,654.83
398 get the dealbook newsletter to make sense of m... 1/7/19 0 $4,025.25
399 tbilisi georgia for three years a windowless w... 1/22/19 1 $3,604.58
400 tehran stepping from a car into a muddy indust... 1/29/19 0 $3,448.12
401 san francisco don t look now storm clouds are ... 2/3/19 0 $3,464.01
402 seoul south korea kim ki won is keeping a secr... 2/10/19 0 $3,690.19
403 in 2017 jamie dimon jpmorgan chase s chief exe... 2/14/19 1 $3,616.88
404 blockchain the ingenious database technology b... 2/15/19 1 $3,620.81
405 this essay has been updated to reflect news de... 2/23/19 1 $4,142.53
406 san francisco some of the world s biggest inte... 2/28/19 1 $3,854.79
407 in october 2017 a swat team descended on james... 3/12/19 1 $3,909.16
408 tokyo a tokyo court handed down a two and a ha... 3/15/19 0 $3,960.91
409 san francisco paul chou was among the many bit... 4/2/19 0 $4,879.88
410 hong kong china is planning new steps that cou... 4/9/19 0 $5,204.96

411 rows × 4 columns