Tune sources

Here we demonstrate included utilities for loading tune data.

from pyabc2 import Tune
from pyabc2.sources import load_example, norbeck, the_session, eskin, bill_black

A few examples are included in the package, accessible with pyabc2.sources.load_example() (returns Tune) and pyabc2.sources.load_example_abc() (returns ABC string).

load_example("For the Love of Music")

Tune(title='For The Love Of Music', key=Gmaj, type='slip jig')

The tune source modules, demonstrated below, download tune data from the internet.

Norbeck

norbeck.load() gives us a list of Tunes for one of Norbeck’s tune type groups (e.g. ‘jigs’, ‘reels’, ‘slip jigs’).

tunes = norbeck.load("jigs")
print(len(tunes), "jigs loaded")

tunes[0]

downloading... 

done

556 jigs loaded

Tune(title="Bride's Favourite, The", key=Gmaj, type='jig')

tunes[-1]

Tune(title='Stone Step, The', key=Gmaj, type='jig')

The Session

the_session.load() gives us a list of Tunes loaded from a (frequently updated) archive of all of the tunes in The Session. This is a large dataset, so here we cap the processing.

tunes = the_session.load(n=500)

tunes[0]

downloading... 

done

/tmp/ipykernel_646/93315575.py:1: UserWarning: 22 out of 500 The Session tune(s) failed to load. Enable logging debug messages to see more info.
  tunes = the_session.load(n=500)

Tune(title="'S Ann An Ìle", key=Gmaj, type='strathspey')

tunes[-1]

Tune(title='A Trip To Galloway', key=Dmaj, type='waltz')

tune = the_session.load_url("https://thesession.org/tunes/21799#setting43712")
tune

Tune(title='The Cherrytree', key=Gmaj, type='jig')

tune.print_measures()

01: G2 d c B G
02: G F G A F D
03: G2 d A G G
04: d e e f g g
05: G2 d c B G
06: G F G A F D
07: B c c g2 B
08: B c g a f d
09: G2 d c B G
10: G F G A F D
11: G2 d A G G
12: d e e f g g
13: G2 d c B G
14: G F G A F D
15: B c c e2 B
16: A G E G A F
17: E B e d B e
18: d e g d B e
19: f B g f B f
20: f g g g a a
21: g a f f g e
22: f g d e d B
23: B d e f g A
24: A B d A G F
25: E B e d B e
26: d e g d B e
27: f B g f B f
28: f g g g a a
29: b2 a a g e
30: f g e e d e
31: d f a e d e
32: d B A A G F

Data archive

The Session data archive (https://github.com/adactio/TheSession-data) has many datasets (pyabc2.sources.the_session.load_meta()), which we can use in other ways besides parsing ABCs to Tunes.

For example, we can look for the most common ABC notes in the corpus.

%%time

df = the_session.load_meta("tunes", convert_dtypes=True)
df

CPU times: user 647 ms, sys: 79.9 ms, total: 726 ms
Wall time: 1.13 s
tune_id setting_id name type meter mode abc date username composer
0 15326 28560 'S Ann An Ìle strathspey 4/4 Gmajor |:G>A B>G c>A B>G|E<E A>G F<D D2|G>A B>G c>A B... 2016-03-31 15:34:45 danninagh <NA>
1 15326 28582 'S Ann An Ìle strathspey 4/4 Gmajor uD2|:{F}v[G,2G2]uB>ud c>A B>G|{D}E2 uA>uG F<D ... 2016-04-03 09:15:08 DonaldK <NA>
2 14625 26955 'S Daor An Tabac reel 4/4 Bminor |:eAAB eABB|eAAB gedB|eAAB eABB|G2AB gedB:|\r\... 2015-07-31 02:47:47 Charles Mackenzie <NA>
3 5478 5478 'S Iomadh Rud A Chunnaic Mi reel 4/4 Gmajor ABBA GEDE|G2AG EGDG|ABBA GEDE|GEDE G2GA|\r\nAB... 2006-02-03 04:45:46 Andy F <NA>
4 5478 11429 'S Iomadh Rud A Chunnaic Mi reel 4/4 Dmajor |:e|f2fe dBAB|dded B2A2|f2fe dBAB|dBAB d2d:|\r... 2011-08-17 00:57:48 malcombpiper <NA>
... ... ... ... ... ... ... ... ... ... ...
53701 11995 11995 Zoidberg's jig 6/8 Dmajor |:f2f f2a|fed B2A|A2A ABd|e2e ~e2e|\r\nf2f f2a... 2012-06-19 12:12:46 DrugCrazed <NA>
53702 22155 44612 Zolloko San Martinak polka 2/4 Fmajor K:F\r\na|:"A" ad' e'f'|"Dm" (3e'e'e' d'a|ac' b... 2022-08-21 12:24:32 Fernando Durbán Galnares Kepa Junkera
53703 11584 11584 Zonaradikos jig 6/8 Dmajor A|:d3 e3|f3f2e|fga gfe|d2c B2A|\r\nd3 e3|f3f2e... 2011-11-14 16:06:26 gian marco <NA>
53704 9013 9013 Zucchini Reel, The reel 4/4 Dmajor Ac|d2d=c ADFA|G2BG =cGBG|Add=c ADFD|GBAG FDAc|... 2008-10-18 06:05:43 Shelley <NA>
53705 13875 24924 Zuppa Inglese jig 6/8 Gmajor |:GAG GAB|cBc cde|d2B GAB|A2G E2D|\r\nGAG GAB|... 2014-10-07 22:55:26 Edward Nunn <NA>

53706 rows × 10 columns

df.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 53706 entries, 0 to 53705
Data columns (total 10 columns):
 #   Column      Non-Null Count  Dtype         
---  ------      --------------  -----         
 0   tune_id     53706 non-null  Int64         
 1   setting_id  53706 non-null  Int64         
 2   name        53706 non-null  string        
 3   type        53706 non-null  category      
 4   meter       53706 non-null  category      
 5   mode        53706 non-null  category      
 6   abc         53706 non-null  string        
 7   date        53706 non-null  datetime64[ns]
 8   username    53706 non-null  string        
 9   composer    17868 non-null  string        
dtypes: Int64(2), category(3), datetime64[ns](1), string(4)
memory usage: 3.1 MB

from pyabc2.note import RE_NOTE as rx

rx

re.compile(r"(?P<acc>\^|\^\^|=|_|__)?(?P<note>[a-gA-G])(?P<oct>[,']*)(?P<num>[0-9]+)?(?P<slash>/+)?(?P<den>[0-9]+)?",
           re.UNICODE)

This regular expression does also match letters in tune titles, say.

["".join(tup) for tup in rx.findall("the quick brown fox jumps over the lazy dog")]

['e', 'c', 'b', 'f', 'e', 'e', 'a', 'd', 'g']

But The Session stores the tune body separately (in the abc field) and encourages a bare-bones melody-focused approach, so we can expect to mostly be matching actual notes.

from pprint import pprint

cool = df.query("tune_id == 1 and setting_id == 1")
display(cool.T)

abc = cool.abc.iloc[0]
print(abc, "\n")

pprint([m.group() for m in rx.finditer(abc)], compact=True)
10131
tune_id 1
setting_id 1
name Cooley's
type reel
meter 4/4
mode Edorian
abc |:D2|EBBA B2 EB|B2 AB dBAG|FDAD BDAD|FDAD dAFD...
date 2001-05-14 18:45:18
username Jeremy
composer <NA> 
|:D2|EBBA B2 EB|B2 AB dBAG|FDAD BDAD|FDAD dAFD|
EBBA B2 EB|B2 AB defg|afec dBAF|DEFD E2:|
|:gf|eB B2 efge|eB B2 gedB|A2 FA DAFA|A2 FA defg|
eB B2 eBgB|eB B2 defg|afec dBAF|DEFD E2:| 

['D2', 'E', 'B', 'B', 'A', 'B2', 'E', 'B', 'B2', 'A', 'B', 'd', 'B', 'A', 'G',
 'F', 'D', 'A', 'D', 'B', 'D', 'A', 'D', 'F', 'D', 'A', 'D', 'd', 'A', 'F', 'D',
 'E', 'B', 'B', 'A', 'B2', 'E', 'B', 'B2', 'A', 'B', 'd', 'e', 'f', 'g', 'a',
 'f', 'e', 'c', 'd', 'B', 'A', 'F', 'D', 'E', 'F', 'D', 'E2', 'g', 'f', 'e',
 'B', 'B2', 'e', 'f', 'g', 'e', 'e', 'B', 'B2', 'g', 'e', 'd', 'B', 'A2', 'F',
 'A', 'D', 'A', 'F', 'A', 'A2', 'F', 'A', 'd', 'e', 'f', 'g', 'e', 'B', 'B2',
 'e', 'B', 'g', 'B', 'e', 'B', 'B2', 'd', 'e', 'f', 'g', 'a', 'f', 'e', 'c',
 'd', 'B', 'A', 'F', 'D', 'E', 'F', 'D', 'E2']

%%time

note_counts = (
    df.abc
    .str.findall(rx)
    .explode()
    .str.join("")
    .value_counts()
)
note_counts

CPU times: user 29.8 s, sys: 404 ms, total: 30.2 s
Wall time: 30.1 s

abc
A       725291
d       663081
B       657368
e       552548
c       448651
         ...  
B4/          1
e'1          1
A2/4         1
e2//         1
B11          1
Name: count, Length: 1016, dtype: int64

note_counts[:20]

abc
A     725291
d     663081
B     657368
e     552548
c     448651
G     444366
f     389584
F     314808
g     302030
E     265911
D     228279
a     202660
A2     96164
d2     93400
B2     79392
G2     74806
b      59546
e2     59518
c2     47039
C      44295
Name: count, dtype: int64

👆 We can see that A (unit duration) is the leader, being a prominent pitch in many of the common keys.

  • 5 in Dmaj

  • 2 in Gmaj

  • 1 in Ador, Amin, Amix, Amaj

Note

A implies A₄, the A above middle C, the A string on a violin, the lower register on the flute, etc.

Note

In general we don’t know the duration of A without context (L: header field, or based on M: if L: is not set). However, in this case, we know that The Session presets the unit duration to 1/8, so A is an eighth note.

from textwrap import wrap

print("\n".join(wrap("  ".join(note_counts[note_counts == 1].index))))

b'2  _A4  D2/3  =c2/3  =A4  E75  =C,  ^F,6  =F,2  =B,6  B,1  D,1  =F,3
=F,,3  E,,3  _B,3/2  =c1  _d6  _G4  ^b4  f'/2  _f4  b,,3  e8/3  A4/3
A,9  _d'3  E5/2  f5/2  =G6  ^G'  A/1  =B/4  G/1  B,//  _a4  e,2  ^g/8
=c'4  _c'  =G8  =e6  F5/4  _c3  =a3  f,3  ^f2/3  ^c2/  A11  e2/4  ^G,,
^A,,  e,,3  e,,2  f,,3  B'/2  F,6  E,8  c3//  D'/  C'/  ^e/2  _c'2
A6//  A,12  ^c'4  ^c'3  =A/4  g,/  d16  =c16  E24  _d/  E33  ^a//  ^C,
=E,2  B32  e23  ^E,2  _e'/  __d  f,8  =D6  _A,6  d6/  ^b/  D11  C6/
^D/2  B,'  B,9  ^A5  G'4  F'3  d14  d'/4  ^A3/4  ^B3/4  A'/  a'4  e4/
f4/  =e//  ^g/2  ^A//  b,/2  D/8  E/8  B/8  G,,4  _g/  ^C,2  =c'3/2
a8/3  c'1/2  C,12  =B5  A7/9  d1/3  a'/  ^G10  ^D6  ^D10  ^D8  ^A10
^a8  E23  A'4  D22  c7  a/3  B16  G16  _a3/2  G,12  _e8  ^c,2  ^B,2
c'3/4  e''  e'3/2  ^f'2  D,'3  F////  F///  E6/  G6/  c9  B7/4  f9/
D''  B4/  e'1  A2/4  e2//  B11

👆 A variety of ABC note specs appear only once. Many of these have unusual durations or accidentals.

What if we ignore everything except the natural note name?

nat_cased_counts = (
    note_counts
    .reset_index(drop=False)
    .rename(columns={"abc": "note"})
    .assign(nat=lambda df: df.note.str.extract(r"([a-gA-G])"))
    .groupby("nat")
    .aggregate({"count": "sum"})["count"]
    .sort_values(ascending=False)
)
nat_cased_counts

nat
A    914323
B    836321
d    818400
e    658063
G    583723
c    571318
f    469408
g    379825
F    378402
E    337970
D    296323
a    248464
b     70739
C     54264
Name: count, dtype: int64

👆 A is still our leader, but otherwise things have shifted a bit. Note C, which generally implies a pitch outside of the range of most whistles and flutes, has the lowest count. Although b is inside that range, many tunes don’t have one.

from pyabc2 import Note

(
    nat_cased_counts
    .to_frame()
    .assign(value=lambda df: df.index.map(lambda x: Note.from_abc(x).value))
    .sort_values("value")["count"]
    .plot.bar(
        xlabel="ABC letters\n(accidentals, octave indicators, and context in key ignored)",
        rot=0,
        ylabel="Count",
        title="ABC prevalance in The Session",
    )
);
../_images/caae55eb43e80b61488a477e2e76f449cc586b135bf8da8182e943b94b1f35cf.png

Eskin

Michael Eskin has tunebooks available at https://michaeleskin.com/tunebooks.html, viewable with his ABC Transcription Tools.

We can load selected tunebooks from there, e.g. the King Street Sessions:

df = eskin.load_meta("kss")
df

downloading... 

done
name abc group
0 The bonniest lass in the world X: 938\nT:The bonniest lass in the world\nR:Re... airs_songs
1 The Brae's of Lochiel X: 939\nT:The Braes of Lochiel\nT:Braigh Loch ... airs_songs
2 Farewell to whiskey X: 940\nT:Farewell to whiskey\nR:Air\nO:Scotla... airs_songs
3 Galen's arrival X: 941\nT:Galen's arrival\nR:Reel\nO:Scotland\... airs_songs
4 Give me your hand X: 942\nT:Give me your hand\nR:Air\nQ:180\nC:R... airs_songs
... ... ... ...
1001 Waiting for Peter X: 933\nT:Waiting for Peter\nR:Waltz\nC:Lee An... waltzes
1002 Waltz of the toys X: 934\nT:Waltz of the toys\nR:Waltz\nC:Michel... waltzes
1003 West of the River Shannon X: 935\nT:West of the River Shannon\nR:Waltz\n... waltzes
1004 Westphalia waltz X: 936\nT:Westphalia waltz\nR:Waltz\nB:The Wal... waltzes
1005 Wind on the heath X: 937\nT:Wind on the heath\nR:Waltz\nO:Scotla... waltzes

1006 rows × 3 columns

df.group.value_counts()

group
reels          353
jigs           260
hornpipes       75
scotchreels     64
polkas          46
slipjigs        37
strathspeys     33
waltzes         29
ocarolan        25
misc_tunes      21
marches         21
slides          19
airs_songs      16
long_dances      7
Name: count, dtype: int64

Tune(df.query("group == 'jigs'").iloc[0].abc)

Tune(title='The academy jig', key=Gmaj, type='Jig')

from IPython.display import display, Markdown

url = "https://michaeleskin.com/abctools/abctools.html?lzw=BoLgUAKiBiD2BOACCALApogMrAbhg8gGaICyArgM4CWAxmAEogUA2VADogFZUDmYAwiExUAXon4BDePFjNmYEiACcAegAcYTCACM6sAGkQAcTBGAogBFEFs0cQBBIwCFEAHwdG7zgCaI0333dzKxs7Rxo3RCc0DCd7F3MzRBBXMB5-PxVCFR4EpxUaFUDEdN80HgAjRAkAJmJ3Uszs3Id8wuL-F28nMKdAtIy0LJy8gqLIxvKq2olIipimnIxankjOxG7e+zdUoA"
display(Markdown(f"<{url}>"))
eskin.load_url(url)

https://michaeleskin.com/abctools/abctools.html?lzw=BoLgUAKiBiD2BOACCALApogMrAbhg8gGaICyArgM4CWAxmAEogUA2VADogFZUDmYAwiExUAXon4BDePFjNmYEiACcAegAcYTCACM6sAGkQAcTBGAogBFEFs0cQBBIwCFEAHwdG7zgCaI0333dzKxs7Rxo3RCc0DCd7F3MzRBBXMB5-PxVCFR4EpxUaFUDEdN80HgAjRAkAJmJ3Uszs3Id8wuL-F28nMKdAtIy0LJy8gqLIxvKq2olIipimnIxankjOxG7e+zdUoA

Tune(title='For The Love Of Music', key=Gmaj, type='slip jig')

Bill Black

Bill Black has an extensive ABC library, available at http://www.capeirish.com/ittl/. We can load all of the tune blocks (strings) with pyabc2.sources.bill_black.load_meta().

abcs = bill_black.load_meta()
len(abcs)

downloading... 

done

10249

Tune(abcs[0])

Tune(title='A-D POLKA', key=Dmaj, type='?')