DRY Résumé Creation with LaTeX, Jinja2, and TOML

Programatically define a way to create, read, update, and delete résumé information.

Specifications

  • One source of data.
  • Facilitate the editing of LaTeX templates into Jinja2 templates via the use of different reserved characters for the Jinja2 language.

System Design

  • "Tom's Obvious, Minimal Language" [TOML] for data organization and storage in text files.
  • LaTeX templates for résumés.
  • Output programs for Portable Document Format [PDF] generation from LaTeX templates.

    pdflatex and xelatex, are LaTeX formats based on the respective engines. All output PDF by default.

  • Python programming language to test and write scripts.

Issues

Maintaining résumé data across various media and platforms is tedious and error prone.

It would be more efficient to maintain one source of data that could be read and then inserted into a given medium.

Quality LaTeX templates are available for download. Editing these templates can also be tedious and error prone. And changing to a new template for a new résumé style means repeating the input of the same information. The Jinja2 templating language is a possible solution to avoiding this repetition.

Nonetheless, it quickly becomes unweidly to use the Jinja2 templating language inside of LaTeX template because of the heavy use of {}% characters in both.

Python code can be used to test and facilitate the process of translation of LaTeX templates into Jinja2 templates and again back to a LaTeX template. This final LaTeX template can be used by output prgrams to generate PDF.

Examples

This LaTeX creates a list in a document.

See Your first LaTeX document tutorial.

\documentclass{article}
\begin{document}
  Hello World!
\begin{itemize}
    \item One
    \item Two
    \item Three
\end{itemize}
\end{document}

This LaTeX template is available here.

It would be ideal to be able to define a data structure with the items and use a templating language to generate the LaTeX template.

Simple Jinja2 template with rendered output

In [16]:
from jinja2 import Template
from faker import Faker

fake = Faker()

context = dict(items=[fake.catch_phrase() for _ in range(3)])
template = Template(
    """Example list in Jinja2
{%- for item in items %}
    - {{ item }}
{%- endfor %}
"""
)
print(template.render(**context))
Example list in Jinja2
    - User-friendly 24hour alliance
    - Front-line zero administration open architecture
    - Secured solution-oriented time-frame

Unwieldy example of attempt to output LaTeX

Everywhere the {} LaTeX characters need preserving require special escaping.

This makes difficult legibility even more difficult.

In [17]:
template = Template(
    r"""Example list in Jinja2
\documentclass{{'{'}}article{{'}'}}
\begin{{'{'}}document{{'}'}}
  Hello World!
\begin{{'{'}}itemize{{'}'}}
{%- for item in items %}
    \item{{ item }}
{%- endfor %}
\end{{'{'}}itemize{{'}'}}
\end{{'{'}}document{{'}'}}
"""
)
print(template.render(**context))
Example list in Jinja2
\documentclass{article}
\begin{document}
  Hello World!
\begin{itemize}
    \itemUser-friendly 24hour alliance
    \itemFront-line zero administration open architecture
    \itemSecured solution-oriented time-frame
\end{itemize}
\end{document}

Possible solution: Python str translation method

  • Redefine the reserved {} characters in a LaTeX template to «»
In [18]:
(latex_translation := str.maketrans((original_latex := {"{": "«", "}": "»"})))
Out[18]:
{123: '«', 125: '»'}
In [19]:
(reverse_latex_translation := str.maketrans(dict(zip(original_latex.values(), original_latex.keys()))))
Out[19]:
{171: '{', 187: '}'}
  • Redefine the {}% characters used in Jinja2 to ≤≥|.
In [20]:
(jina_translation := str.maketrans((original_jinja := {"≤": "{", "≥": "}", "|": "%",})))
Out[20]:
{8804: '{', 8805: '}', 124: '%'}
In [21]:
(
    reverse_jinja_translation := str.maketrans(
        dict(zip(original_jinja.values(), original_jinja.keys()))
    )
)
Out[21]:
{123: '≤', 125: '≥', 37: '|'}
In [22]:
print(
    r"""{%- for item in items %}
    \item{{ item }}
{%- endfor %}""".translate(
        reverse_jinja_translation
    )
)
≤|- for item in items |≥
    \item≤≤ item ≥≥
≤|- endfor |≥
In [23]:
print(
    latex_template := Template(
        r"""\documentclass{article}
\begin{document}
  Hello World!
\begin{itemize}
    ≤|- for item in items |≥
    \item ≤≤ item ≥≥
≤|- endfor |≥
\end{itemize}
\end{document}""".translate(
            latex_translation
        )
        .translate(jina_translation)
        .translate(reverse_latex_translation)
    ).render(**context)
)
\documentclass{article}
\begin{document}
  Hello World!
\begin{itemize}
    \item User-friendly 24hour alliance
    \item Front-line zero administration open architecture
    \item Secured solution-oriented time-frame
\end{itemize}
\end{document}

Write the latex template string to a tempfile

In [24]:
import tempfile
import os

fd, name = tempfile.mkstemp()
with os.fdopen(fd, 'w') as fh:
    fh.write(latex_template)

Upload the file to Minio object server.

In [25]:
!mc cp $name dokkuminio/public/simple_list.tex
...p2wi7fndg:  246 B / 246 B ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 100.00% 20.32 KB/s 0s

pipe the output into pdflatex

In [26]:
!curl -s https://minio.apps.selfip.com/public/simple_list.tex | pdflatex -output-directory latex_templates/
This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
 restricted \write18 enabled.
**entering extended mode
LaTeX2e <2017-04-15>
Babel <3.18> and hyphenation patterns for 3 language(s) loaded.

*(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))
(latex_templates//texput.aux)
*
*
*(/usr/share/texlive/texmf-dist/tex/latex/base/omscmr.fd)
*
*
*
*[1{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}]
(latex_templates//texput.aux)</usr/share/texlive/texmf-dist/fonts/type1/public/
amsfonts/cm/cmr10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfont
s/cm/cmsy10.pfb>
Output written on latex_templates//texput.pdf (1 page, 23107 bytes).
Transcript written on latex_templates//texput.log.

Upload the PDF output to Dokku self-hosted Minio S3-compatible object storage

In [27]:
!mc cp latex_templates/texput.pdf dokkuminio/public/
...exput.pdf:  22.57 KB / 22.57 KB ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 100.00% 1.41 MB/s 0s

Display the PDF created.

In [28]:
from IPython.display import display, IFrame
In [30]:
IFrame("https://minio.apps.selfip.com/public/texput.pdf", width=900, height=800)
Out[30]:

Real Python Matplotlib Tutorial

In [56]:
import matplotlib.pyplot as plt
In [57]:
import numpy as np
In [58]:
np.random.seed(444)
  • One important big-picture matplotlib concept is its object hierarchy.

  • A “hierarchy” here means that there is a tree-like structure of matplotlib objects underlying each plot.

  • A Figure object is the outermost container for a matplotlib graphic, which can contain multiple Axes objects. One source of confusion is the name: an Axes actually translates into what we think of as an individual plot or graph (rather than the plural of “axis,” as we might expect).

In [59]:
fig, _ = plt.subplots()
In [60]:
type(one_tick := fig.axes[0].yaxis.get_major_ticks()[0])
Out[60]:
matplotlib.axis.YTick
  • Notice that we didn’t pass arguments to subplots() here. The default call is subplots(nrows=1, ncols=1)

In [61]:
fig, ax = plt.subplots()
In [62]:
type(ax)
Out[62]:
matplotlib.axes._subplots.AxesSubplot
  • We can call its instance methods to manipulate the plot similarly to how we call pyplots functions. Let’s illustrate with a stacked area graph of three time series

In [63]:
(rng := np.arange(50))
Out[63]:
array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
       17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
       34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49])
In [64]:
(rnd := np.random.randint(0, 10, size=(3, rng.size)))
Out[64]:
array([[3, 0, 7, 8, 3, 4, 7, 6, 8, 9, 2, 2, 2, 0, 3, 8, 0, 6, 6, 0, 3, 0,
        6, 7, 9, 3, 8, 7, 3, 2, 6, 9, 2, 9, 8, 9, 3, 2, 2, 8, 1, 5, 6, 7,
        6, 0, 0, 0, 0, 4],
       [8, 1, 9, 8, 5, 8, 9, 4, 6, 6, 4, 1, 8, 2, 7, 9, 3, 4, 2, 5, 0, 0,
        8, 1, 0, 9, 9, 3, 2, 7, 6, 0, 5, 5, 4, 8, 3, 4, 9, 4, 7, 1, 5, 4,
        4, 0, 2, 2, 5, 8],
       [5, 6, 6, 1, 1, 6, 8, 4, 1, 0, 9, 2, 3, 7, 3, 3, 2, 7, 8, 6, 6, 7,
        5, 7, 3, 9, 1, 3, 0, 4, 7, 5, 1, 5, 1, 4, 9, 7, 2, 4, 3, 7, 9, 2,
        2, 0, 1, 5, 2, 4]])
In [65]:
(yrs := 1950 + rng)
Out[65]:
array([1950, 1951, 1952, 1953, 1954, 1955, 1956, 1957, 1958, 1959, 1960,
       1961, 1962, 1963, 1964, 1965, 1966, 1967, 1968, 1969, 1970, 1971,
       1972, 1973, 1974, 1975, 1976, 1977, 1978, 1979, 1980, 1981, 1982,
       1983, 1984, 1985, 1986, 1987, 1988, 1989, 1990, 1991, 1992, 1993,
       1994, 1995, 1996, 1997, 1998, 1999])
In [66]:
fig, ax = plt.subplots(figsize=(5, 3))
In [67]:
ax.stackplot(yrs, rng + rnd, labels=["Eastasia", "Eurasia", "Oceania"])
Out[67]:
[<matplotlib.collections.PolyCollection at 0x7ff3b986e640>,
 <matplotlib.collections.PolyCollection at 0x7ff3b9842c30>,
 <matplotlib.collections.PolyCollection at 0x7ff3b981aaf0>]
In [68]:
ax.set_title("Combined debt growth over time")
Out[68]:
Text(0.5, 1, 'Combined debt growth over time')
In [69]:
ax.legend(loc="upper left")
Out[69]:
<matplotlib.legend.Legend at 0x7ff3b97ccfa0>
In [70]:
ax.set_ylabel("Total debt")
Out[70]:
Text(3.200000000000003, 0.5, 'Total debt')
In [71]:
ax.set_xlim(xmin=yrs[0], xmax=yrs[-1])
Out[71]:
(1950, 1999)
In [72]:
fig.tight_layout()
In [73]:
fig
Out[73]:

Let’s look at an example with multiple subplots (Axes) within one Figure, plotting two correlated arrays that are drawn from the discrete uniform distribution:

In [74]:
(x := np.random.randint(low=1, high=11, size=50))
Out[74]:
array([ 9,  1,  5,  6, 10,  9,  7,  7, 10,  6,  8,  6,  4,  9,  3,  7,  6,
        9,  2, 10,  7,  2,  2,  5,  7,  9,  5,  9,  9,  8,  6,  3,  4,  3,
        1,  1,  5,  7,  6,  4,  4,  1,  9,  5, 10,  3,  5,  4,  1,  9])
In [75]:
(y := x + np.random.randint(1, 5, size=x.size))
Out[75]:
array([11,  5,  6,  7, 14, 13,  8, 10, 11,  8, 10,  7,  6, 11,  6, 10,  8,
       13,  6, 11, 10,  6,  6,  9,  9, 13,  8, 12, 12, 11,  9,  4,  6,  5,
        4,  2,  9,  8,  7,  8,  6,  3, 13,  8, 12,  4,  9,  7,  4, 11])
In [76]:
(data := np.column_stack((x, y)))
Out[76]:
array([[ 9, 11],
       [ 1,  5],
       [ 5,  6],
       [ 6,  7],
       [10, 14],
       [ 9, 13],
       [ 7,  8],
       [ 7, 10],
       [10, 11],
       [ 6,  8],
       [ 8, 10],
       [ 6,  7],
       [ 4,  6],
       [ 9, 11],
       [ 3,  6],
       [ 7, 10],
       [ 6,  8],
       [ 9, 13],
       [ 2,  6],
       [10, 11],
       [ 7, 10],
       [ 2,  6],
       [ 2,  6],
       [ 5,  9],
       [ 7,  9],
       [ 9, 13],
       [ 5,  8],
       [ 9, 12],
       [ 9, 12],
       [ 8, 11],
       [ 6,  9],
       [ 3,  4],
       [ 4,  6],
       [ 3,  5],
       [ 1,  4],
       [ 1,  2],
       [ 5,  9],
       [ 7,  8],
       [ 6,  7],
       [ 4,  8],
       [ 4,  6],
       [ 1,  3],
       [ 9, 13],
       [ 5,  8],
       [10, 12],
       [ 3,  4],
       [ 5,  9],
       [ 4,  7],
       [ 1,  4],
       [ 9, 11]])
In [77]:
fig, (ax1, ax2) = plt.subplots(nrows=1, ncols=2, figsize=(8, 4))
In [78]:
ax1.scatter(x=x, y=y, marker="o", c="r", edgecolor="b")
Out[78]:
<matplotlib.collections.PathCollection at 0x7ff3bb7f7a00>
In [79]:
ax1.set_title("Scatter: $x$ versus $y$")
Out[79]:
Text(0.5, 1, 'Scatter: $x$ versus $y$')
In [80]:
ax1.set_xlabel("$x$")
Out[80]:
Text(0.5, 3.1999999999999993, '$x$')
In [81]:
ax1.set_ylabel("$y$")
Out[81]:
Text(3.200000000000003, 0.5, '$y$')
In [82]:
ax2.hist(data, bins=np.arange(data.min(), data.max()), label=("x", "y"))
Out[82]:
([array([5., 3., 4., 5., 6., 6., 6., 2., 9., 4., 0., 0.]),
  array([0., 1., 1., 4., 2., 8., 4., 7., 5., 4., 6., 7.])],
 array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13]),
 <a list of 2 Lists of Patches objects>)
In [83]:
ax2.legend(loc=(0.65, 0.8))
Out[83]:
<matplotlib.legend.Legend at 0x7ff3bb794dc0>
In [84]:
ax2.set_title("Frequencies of $x$ and $y$")
Out[84]:
Text(0.5, 1, 'Frequencies of $x$ and $y$')
In [85]:
ax2.yaxis.tick_right()
  • Text inside dollar signs utilizes TeXmarkup to put variables in italics.

  • Because we’re creating a “1x2” Figure, the returned result of plt.subplots(1, 2) is now a Figure object and a NumPy array of Axes objects. (You can inspect this with fig, axs = plt.subplots(1, 2) and taking a look at axs.)

In [86]:
fig
Out[86]:
In [87]:
tuple(fig.axes[i] is ax for i, ax in zip(range(2), (ax1, ax2)))
Out[87]:
(True, True)
  • Taking this one step further, we could alternatively create a figure that holds a 2x2 grid of Axes objects

In [88]:
fig, ax = plt.subplots(nrows=2, ncols=2, figsize=(7, 7))
  • Now, what is ax? It’s no longer a single Axes, but a two-dimensional NumPy array of them

In [89]:
type(ax)
Out[89]:
numpy.ndarray
In [90]:
ax
Out[90]:
array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7ff3b97b0a00>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7ff3b975d3c0>],
       [<matplotlib.axes._subplots.AxesSubplot object at 0x7ff3b971a730>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7ff3b97da410>]],
      dtype=object)
In [91]:
ax1, ax2, ax3, ax4 = ax.flatten()

To illustrate some more advanced subplot features, let’s pull some macroeconomic California housing data extracted from a compressed tar archive, using io, tarfile, and urllib from Python’s Standard Library.

In [92]:
import tarfile
from urllib.request import urlretrieve
from pathlib import Path
In [93]:
filepath, response = urlretrieve(
    "http://www.dcc.fc.up.pt/~ltorgo/Regression/cal_housing.tgz"
)
In [94]:
filepath
Out[94]:
'/tmp/tmpyh987137'
In [95]:
with tarfile.open(name=filepath, mode="r") as archive:
    housing = np.loadtxt(
        archive.extractfile("CaliforniaHousing/cal_housing.data"), delimiter=","
    )
In [96]:
housing
Out[96]:
array([[-1.2223e+02,  3.7880e+01,  4.1000e+01, ...,  1.2600e+02,
         8.3252e+00,  4.5260e+05],
       [-1.2222e+02,  3.7860e+01,  2.1000e+01, ...,  1.1380e+03,
         8.3014e+00,  3.5850e+05],
       [-1.2224e+02,  3.7850e+01,  5.2000e+01, ...,  1.7700e+02,
         7.2574e+00,  3.5210e+05],
       ...,
       [-1.2122e+02,  3.9430e+01,  1.7000e+01, ...,  4.3300e+02,
         1.7000e+00,  9.2300e+04],
       [-1.2132e+02,  3.9430e+01,  1.8000e+01, ...,  3.4900e+02,
         1.8672e+00,  8.4700e+04],
       [-1.2124e+02,  3.9370e+01,  1.6000e+01, ...,  5.3000e+02,
         2.3886e+00,  8.9400e+04]])
In [97]:
(y := housing[:, -1])
Out[97]:
array([452600., 358500., 352100., ...,  92300.,  84700.,  89400.])

The property T is an accessor to the method transpose().

In [98]:
pop, age = housing[:, [4, 7]].T
In [99]:
pop, age
Out[99]:
(array([ 129., 1106.,  190., ...,  485.,  409.,  616.]),
 array([8.3252, 8.3014, 7.2574, ..., 1.7   , 1.8672, 2.3886]))

Next let’s define a “helper function” that places a text box inside of a plot and acts as an “in-plot title”:

In [100]:
def add_titlebox(ax, text):
    ax.text(
        0.55,
        0.8,
        text,
        horizontalalignment="center",
        transform=ax.transAxes,
        bbox=dict(facecolor="white", alpha=0.6),
        fontsize=12.5,
    )
    return ax
In [101]:
gridsize = (3, 2)
fig = plt.figure(figsize=(12, 8))
ax1 = plt.subplot2grid(gridsize, (0, 0), colspan=2, rowspan=2)
ax2 = plt.subplot2grid(gridsize, (2, 0))
ax3 = plt.subplot2grid(gridsize, (2, 1))
In [102]:
ax1.set_title("Home value as a function of home age & area population", fontsize=14)
sctr = ax1.scatter(x=age, y=pop, c=y, cmap="RdYlGn")
plt.colorbar(sctr, ax=ax1, format="$%d")
ax1.set_yscale("log")
ax2.hist(age, bins="auto")
ax3.hist(pop, bins="auto", log=True)
add_titlebox(ax2, "Histogram: home age")
add_titlebox(ax3, "Histogram: area population (log scl.)")
Out[102]:
<matplotlib.axes._subplots.AxesSubplot at 0x7ff3b8a668c0>
In [103]:
fig
Out[103]:
In [104]:
[plt.figure(i) for i in plt.get_fignums()]
Out[104]:
[]
In [105]:
plt.get_fignums()
Out[105]:
[]

While ax.plot() is one of the most common plotting methods on an Axes, there are a whole host of others, as well. (We used ax.stackplot() above. You can find the complete list here.)

Methods that get heavy use are imshow() and matshow(), with the latter being a wrapper around the former. These are useful anytime that a raw numerical array can be visualized as a colored grid.

In [106]:
(x := np.diag(np.arange(2, 12))[::-1])
Out[106]:
array([[ 0,  0,  0,  0,  0,  0,  0,  0,  0, 11],
       [ 0,  0,  0,  0,  0,  0,  0,  0, 10,  0],
       [ 0,  0,  0,  0,  0,  0,  0,  9,  0,  0],
       [ 0,  0,  0,  0,  0,  0,  8,  0,  0,  0],
       [ 0,  0,  0,  0,  0,  7,  0,  0,  0,  0],
       [ 0,  0,  0,  0,  6,  0,  0,  0,  0,  0],
       [ 0,  0,  0,  5,  0,  0,  0,  0,  0,  0],
       [ 0,  0,  4,  0,  0,  0,  0,  0,  0,  0],
       [ 0,  3,  0,  0,  0,  0,  0,  0,  0,  0],
       [ 2,  0,  0,  0,  0,  0,  0,  0,  0,  0]])
In [107]:
x[np.diag_indices_from(x[::-1])] = np.arange(2, 12)
x
Out[107]:
array([[ 2,  0,  0,  0,  0,  0,  0,  0,  0, 11],
       [ 0,  3,  0,  0,  0,  0,  0,  0, 10,  0],
       [ 0,  0,  4,  0,  0,  0,  0,  9,  0,  0],
       [ 0,  0,  0,  5,  0,  0,  8,  0,  0,  0],
       [ 0,  0,  0,  0,  6,  7,  0,  0,  0,  0],
       [ 0,  0,  0,  0,  6,  7,  0,  0,  0,  0],
       [ 0,  0,  0,  5,  0,  0,  8,  0,  0,  0],
       [ 0,  0,  4,  0,  0,  0,  0,  9,  0,  0],
       [ 0,  3,  0,  0,  0,  0,  0,  0, 10,  0],
       [ 2,  0,  0,  0,  0,  0,  0,  0,  0, 11]])
In [109]:
(x2 := np.arange(x.size).reshape(x.shape))
Out[109]:
array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
       [20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
       [30, 31, 32, 33, 34, 35, 36, 37, 38, 39],
       [40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
       [50, 51, 52, 53, 54, 55, 56, 57, 58, 59],
       [60, 61, 62, 63, 64, 65, 66, 67, 68, 69],
       [70, 71, 72, 73, 74, 75, 76, 77, 78, 79],
       [80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
       [90, 91, 92, 93, 94, 95, 96, 97, 98, 99]])
In [110]:
sides = (
    "left",
    "right",
    "top",
    "bottom",
)
In [111]:
nolabels = {s: False for s in sides}
In [113]:
nolabels.update({f"label{s}": False for s in sides})
In [114]:
nolabels
Out[114]:
{'left': False,
 'right': False,
 'top': False,
 'bottom': False,
 'labelleft': False,
 'labelright': False,
 'labeltop': False,
 'labelbottom': False}
In [115]:
from mpl_toolkits.axes_grid1.axes_divider import make_axes_locatable
In [122]:
with plt.rc_context(rc={"axes.grid": False}):
    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(8, 4))
    ax1.matshow(x)
    img2 = ax2.matshow(x2, cmap="RdYlGn_r")
    for ax in (ax1, ax2):
        ax.tick_params(axis="both", which="both", **nolabels)
    for i, j in zip(*x.nonzero()):
        ax1.text(j, i, x[i, j], color="white", ha="center", va="center")
    divider = make_axes_locatable(ax2)
    cax = divider.append_axes("right", size="5%", pad=0)
    plt.colorbar(img2, cax=cax, ax=[ax1, ax2])
    fig.suptitle("Heatmaps with `Axes.matshow`", fontsize=16)

The pandas library has become popular for not just for enabling powerful data analysis, but also for its handy pre-canned plotting methods. Interestingly though, pandas plotting methods are really just convenient wrappers around existing matplotlib calls.

In [123]:
import pandas as pd
In [124]:
(s := pd.Series(np.arange(5), index=list("abcde")))
Out[124]:
a    0
b    1
c    2
d    3
e    4
dtype: int64
In [125]:
(ax := s.plot())
Out[125]:
<matplotlib.axes._subplots.AxesSubplot at 0x7ff3adf9ef50>
In [126]:
type(ax)
Out[126]:
matplotlib.axes._subplots.AxesSubplot
In [128]:
id(plt.gca()), id(ax)
Out[128]:
(140684556626192, 140684572618576)
In [129]:
import matplotlib.transforms as mtransforms
In [179]:
url = "https://fred.stlouisfed.org/graph/fredgraph.csv?id=VIXCLS"
In [186]:
vix = pd.read_csv(
    url, index_col=0, parse_dates=True, infer_datetime_format=True, squeeze=True
).dropna()
In [188]:
vix = vix[vix != "."]
In [189]:
(ma := vix.rolling("90d").mean())
Out[189]:
DATE
1990-01-02    17.240000
1990-01-03    17.715000
1990-01-04    18.216667
1990-01-05    18.690000
1990-01-08    19.004000
                ...    
2020-02-21    14.044500
2020-02-24    14.310000
2020-02-25    14.582881
2020-02-26    14.799167
2020-02-27    15.241500
Name: VIXCLS, Length: 7595, dtype: float64
In [193]:
{type(item) for item in vix}
Out[193]:
{str}
In [222]:
vix_ = pd.Series([float(item) for item in vix])
In [223]:
state = pd.cut(ma, bins=[-np.inf, 14, 18, 24, np.inf], labels=range(4))
cmap = plt.get_cmap("RdYlGn_r")
ma.plot(color="black", linewidth=1.5, marker="", figsize=(8, 4), label="VIX 90d MA")
ax = plt.gca()  # Get the current Axes that ma.plot() references
ax.set_ylabel("90d moving average: CBOE VIX")
ax.set_title("Volatility Regime State")
ax.grid(False)
ax.legend(loc="upper center")
ax.set_xlim(xmin=ma.index[0], xmax=ma.index[-1])
trans = mtransforms.blended_transform_factory(ax.transData, ax.transAxes)
for i, color in enumerate(cmap([0.2, 0.4, 0.6, 0.8])):
    ax.fill_between(ma.index, 0, 1, where=state == i, facecolor=color, transform=trans)
ax.axhline(
    vix_.mean(),
    linestyle="dashed",
    color="xkcd:dark grey",
    alpha=0.6,
    label="Full-period mean",
    marker="",
)
Out[223]:
<matplotlib.lines.Line2D at 0x7ff3ac632370>

Use JSON Web Tokens

Use JSON web tokens.

In [1]:
import jwt

jwt.encode?

Signature:
jwt.encode(
    payload,
    key,
    algorithm='HS256',
    headers=None,
    json_encoder=None,
)
Type:      method
In [2]:
(encoded := jwt.encode({"some": "payload"}, "secret"))
Out[2]:
b'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Joh1R2dYzkRvDkqv3sygm5YyK8Gi4ShZqbhK2gxcs2U'
In [3]:
jwt.decode(encoded, "secret", algorithms=["HS256"])
Out[3]:
{'some': 'payload'}

Set an expiration date.

In [4]:
from datetime import datetime, timedelta
import time
import arrow
In [5]:
(
    encoded := jwt.encode(
        {"exp": (exp := datetime.utcnow() + timedelta(seconds=3))}, "secret"
    )
)
Out[5]:
b'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1Nzc3MDg2ODV9.3sInaD8S16T9Iva3I-OI0-4BtXbxh7PGIkHfepXNGGQ'
In [6]:
print(len(encoded))
105
In [7]:
time.sleep(4)  # Allow time to expire.
In [8]:
try:
    jwt.decode(encoded, "secret")
except jwt.ExpiredSignatureError:
    print(f"Signature expired {arrow.get(exp).humanize()}.")
Signature expired just now.

Insert a Menu and Anchor Tags in a Long Jupyter Notebook Output Cell

Download a previously stored dataframe

In [1]:
from pathlib import Path
import urllib.request
from urllib import parse
import pickle
from string import digits
from functools import partial

from IPython.display import HTML, Image, Markdown
In [2]:
(df,) = (
    pickle.loads(Path(fp).read_bytes())
    for fp, _ in (
        urllib.request.urlretrieve(
            (Path.home() / ".texpander" / "iowa_sports_pk_url").read_text()
        ),
    )
)

Display data frame

In [3]:
display(df)
xpath url sport sport_id sex
0 /html/body/form/div/div[3]/table/tr[2]/td[1]/t... http://quikstatsiowa.com/Public/BBSB/TeamStand... Baseball B25923B5-D303-41CA-B9B3-DF2527D84CDD boys
1 /html/body/form/div/div[3]/table/tr[2]/td[1]/t... http://quikstatsiowa.com/Public/Basketball/Tea... Basketball 57C38F60-B323-4087-A557-9ED925DC546D boys
2 /html/body/form/div/div[3]/table/tr[2]/td[2]/t... http://quikstatsiowa.com/Public/Basketball/Tea... Basketball B657ECDF-ECD0-4429-810A-9F9274EC4AAA girls
3 /html/body/form/div/div[3]/table/tr[2]/td[1]/t... http://quikstatsiowa.com/Public/Bowling/TeamSt... Bowling DA3506E8-E4CA-4175-BF69-BEBBDC2FD878 boys
4 /html/body/form/div/div[3]/table/tr[2]/td[2]/t... http://quikstatsiowa.com/Public/Bowling/TeamSt... Bowling 0C6DFBCF-98C4-4B01-9F56-17B02E9E47E1 girls
5 /html/body/form/div/div[3]/table/tr[2]/td[1]/t... http://quikstatsiowa.com/Public/Golf/TeamStand... Fall Golf 92A34DE4-ACB3-4282-BF29-571A97DE1946 boys
6 /html/body/form/div/div[3]/table/tr[2]/td[1]/t... http://quikstatsiowa.com/Public/Football/TeamS... Football 91A308DE-5763-4DAA-8C03-9AF66611E0BC boys
7 /html/body/form/div/div[3]/table/tr[2]/td[2]/t... http://quikstatsiowa.com/Public/Golf/TeamStand... Golf 6DC124A1-D8C4-4F88-84EF-5C6B4FD4A688 girls
8 /html/body/form/div/div[3]/table/tr[2]/td[1]/t... http://quikstatsiowa.com/Public/Soccer/TeamSta... Soccer 9D4214D2-EBE6-429E-9005-C11D2A29C89B boys
9 /html/body/form/div/div[3]/table/tr[2]/td[2]/t... http://quikstatsiowa.com/Public/Soccer/TeamSta... Soccer 65E5DA09-90C6-45F5-847A-F9A84FD9C5B0 girls
10 /html/body/form/div/div[3]/table/tr[2]/td[2]/t... http://quikstatsiowa.com/Public/BBSB/TeamStand... Softball D97DD7D0-0BEF-404A-B041-7E51ACFDBD16 girls
11 /html/body/form/div/div[3]/table/tr[2]/td[1]/t... http://quikstatsiowa.com/Public/Golf/TeamStand... Spring Golf FC614ADE-B5DA-4012-A95E-0FD2A594FE9D boys
12 /html/body/form/div/div[3]/table/tr[2]/td[1]/t... http://quikstatsiowa.com/Public/Swimming/Indiv... Swimming 139DCB57-4343-4FB8-BAF9-970E5D64597F boys
13 /html/body/form/div/div[3]/table/tr[2]/td[2]/t... http://quikstatsiowa.com/Public/Swimming/Indiv... Swimming 71F7113B-576F-4372-9E9B-4C746F251946 girls
14 /html/body/form/div/div[3]/table/tr[2]/td[1]/t... http://quikstatsiowa.com/Public/Tennis/TeamSta... Tennis 19786FF3-ADA3-4C7A-A94F-FAC0811118F5 boys
15 /html/body/form/div/div[3]/table/tr[2]/td[2]/t... http://quikstatsiowa.com/Public/Tennis/TeamSta... Tennis 6086C2DF-4661-4701-BFF1-3BB32C081B88 girls
16 /html/body/form/div/div[3]/table/tr[2]/td[1]/t... http://quikstatsiowa.com/Public/Track/Individu... Track & Field EB178641-26F1-464D-97F1-A1D101AE35D6 boys
17 /html/body/form/div/div[3]/table/tr[2]/td[2]/t... http://quikstatsiowa.com/Public/Track/Individu... Track & Field 93AAC882-9E72-4621-B16F-95F389BA7F15 girls
18 /html/body/form/div/div[3]/table/tr[2]/td[2]/t... http://quikstatsiowa.com/Public/Volleyball/Tea... Volleyball 83298383-D7D7-4670-9C6B-24DDB8B2E773 girls
In [4]:
from chamelboots import ChameleonTemplate as CT
from chamelboots import TalStatement as TS
from chamelboots.constants import FAKE, JoinWith
from chamelboots.html.utils import prettify_html

Define TAL statements

In [15]:
TSCS, TSR, TSA = (
    TS(*args)
    for args in (
        ("content", f"structure content"),
        ("repeat", "content items"),
        ("attributes", "attributes"),
    )
)
In [14]:
LINK = CT("a", (TSCS, TSA)).render
SPAN = partial(
    CT("span", (TSCS, TSA)).render, attributes=dict(style="font-size: 2.5rem;")
)
SPAN(content="foo")
Out[14]:
'<span style="font-size: 2.5rem;">foo</span>'

Create anchor tags

An html id cannot start with digits so strip them.

In [7]:
menu_items, anchors = zip(
    *(
        (
            LINK(
                content=f"{item.sex}' {item.sport}",
                attributes={
                    "href": f"#{(id_ := item.sport_id.strip(digits))}",
                    "id": (menu_id := f"menu-{id_}"),
                },
            ),
            LINK(
                content=SPAN(content="back to menu"),
                attributes={"id": id_, "href": f"#{menu_id}"},
            ),
        )
        for item in df.itertuples()
    )
)
In [8]:
list_items = prettify_html(
    CT("ul", (TSCS,)).render(content=CT("li", (TSR, TSCS)).render(items=menu_items))
)

Display truncated portion of HTML

In [9]:
print(JoinWith.LINES(list_items.splitlines()[:10]))
<ul>
 <li>
  <a href="#B25923B5-D303-41CA-B9B3-DF2527D84CDD" id="menu-B25923B5-D303-41CA-B9B3-DF2527D84CDD">
   boys' Baseball
  </a>
 </li>
 <li>
  <a href="#C38F60-B323-4087-A557-9ED925DC546D" id="menu-C38F60-B323-4087-A557-9ED925DC546D">
   boys' Basketball
  </a>

Anchors

In [10]:
anchors[:5]
Out[10]:
('<a id="B25923B5-D303-41CA-B9B3-DF2527D84CDD" href="#menu-B25923B5-D303-41CA-B9B3-DF2527D84CDD"><span style="font-size: 2.5rem;">back to menu</span></a>',
 '<a id="C38F60-B323-4087-A557-9ED925DC546D" href="#menu-C38F60-B323-4087-A557-9ED925DC546D"><span style="font-size: 2.5rem;">back to menu</span></a>',
 '<a id="B657ECDF-ECD0-4429-810A-9F9274EC4AAA" href="#menu-B657ECDF-ECD0-4429-810A-9F9274EC4AAA"><span style="font-size: 2.5rem;">back to menu</span></a>',
 '<a id="DA3506E8-E4CA-4175-BF69-BEBBDC2FD" href="#menu-DA3506E8-E4CA-4175-BF69-BEBBDC2FD"><span style="font-size: 2.5rem;">back to menu</span></a>',
 '<a id="C6DFBCF-98C4-4B01-9F56-17B02E9E47E" href="#menu-C6DFBCF-98C4-4B01-9F56-17B02E9E47E"><span style="font-size: 2.5rem;">back to menu</span></a>')

Display list of links.

Display scaled and cropped screenshots of each website page

In [12]:
from chamelboots.imageutils import get_scaled_screenshot
In [13]:
for anchor, item in zip(anchors, df.itertuples()):
    for item in (HTML(anchor), Image(filename=get_scaled_screenshot(item.url))):
        display(item)