-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathpython-shims.html
More file actions
342 lines (339 loc) · 24.8 KB
/
Copy pathpython-shims.html
File metadata and controls
342 lines (339 loc) · 24.8 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
<head>
<meta charset="utf-8" />
<meta name="generator" content="pandoc" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
<title>Intercepting Python Programs Without Modifying Them</title>
<style>
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
div.columns{display: flex; gap: min(4vw, 1.5em);}
div.column{flex: auto; overflow-x: auto;}
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
/* The extra [class] is a hack that increases specificity enough to
override a similar rule in reveal.js */
ul.task-list[class]{list-style: none;}
ul.task-list li input[type="checkbox"] {
font-size: inherit;
width: 0.8em;
margin: 0 0.8em 0.2em -1.6em;
vertical-align: middle;
}
/* CSS for syntax highlighting */
html { -webkit-text-size-adjust: 100%; }
pre > code.sourceCode { white-space: pre; position: relative; }
pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
pre > code.sourceCode > span:empty { height: 1.2em; }
.sourceCode { overflow: visible; }
code.sourceCode > span { color: inherit; text-decoration: inherit; }
div.sourceCode { margin: 1em 0; }
pre.sourceCode { margin: 0; }
@media screen {
div.sourceCode { overflow: auto; }
}
@media print {
pre > code.sourceCode { white-space: pre-wrap; }
pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; }
}
pre.numberSource code
{ counter-reset: source-line 0; }
pre.numberSource code > span
{ position: relative; left: -4em; counter-increment: source-line; }
pre.numberSource code > span > a:first-child::before
{ content: counter(source-line);
position: relative; left: -1em; text-align: right; vertical-align: baseline;
border: none; display: inline-block;
-webkit-touch-callout: none; -webkit-user-select: none;
-khtml-user-select: none; -moz-user-select: none;
-ms-user-select: none; user-select: none;
padding: 0 4px; width: 4em;
color: #aaaaaa;
}
pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; }
div.sourceCode
{ }
@media screen {
pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }
}
code span.al { color: #ff0000; font-weight: bold; } /* Alert */
code span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
code span.at { color: #7d9029; } /* Attribute */
code span.bn { color: #40a070; } /* BaseN */
code span.bu { color: #008000; } /* BuiltIn */
code span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
code span.ch { color: #4070a0; } /* Char */
code span.cn { color: #880000; } /* Constant */
code span.co { color: #60a0b0; font-style: italic; } /* Comment */
code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
code span.do { color: #ba2121; font-style: italic; } /* Documentation */
code span.dt { color: #902000; } /* DataType */
code span.dv { color: #40a070; } /* DecVal */
code span.er { color: #ff0000; font-weight: bold; } /* Error */
code span.ex { } /* Extension */
code span.fl { color: #40a070; } /* Float */
code span.fu { color: #06287e; } /* Function */
code span.im { color: #008000; font-weight: bold; } /* Import */
code span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
code span.kw { color: #007020; font-weight: bold; } /* Keyword */
code span.op { color: #666666; } /* Operator */
code span.ot { color: #007020; } /* Other */
code span.pp { color: #bc7a00; } /* Preprocessor */
code span.sc { color: #4070a0; } /* SpecialChar */
code span.ss { color: #bb6688; } /* SpecialString */
code span.st { color: #4070a0; } /* String */
code span.va { color: #19177c; } /* Variable */
code span.vs { color: #4070a0; } /* VerbatimString */
code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */
</style>
<link rel="stylesheet" href="styles.css" />
<link rel="icon" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>🚀</text></svg>" />
</head>
<body>
<p><em>I am currently working on loading large language models fast. In
this context, I needed to intercept part of a library’s internals
without modifying its code. This got me looking into how Python’s
<code>site</code> module and <code>.pth</code> files work, and how they
can be used to hook into arbitrary code. In this post, I document what I
learned.</em></p>
<h1
id="intercepting-python-programs-without-modifying-them">Intercepting
Python Programs Without Modifying Them</h1>
<p><a href="/" class="home-button">🏠 Home</a></p>
<!-- We create a virtual environment:
```sh
python3 -m venv venv
```
Then check what's in `site-packages`:
```
ls venv/lib/python3.14/site-packages
pip pip-25.3.dist-info
```
And find where `site-packages` lives:
```
venv/bin/python -c 'import site; print(site.getsitepackages()[0])'
/Users/youssefboughizane/Documents/projects/shim/venv/lib/python3.14/site-packages
``` -->
<p><em>Shims</em> are a way to intercept and modify the behavior of
existing code without changing it. In Python, we can use
<code>.pth</code> files to create shims that are automatically imported
when Python starts. This allows us to hook into arbitrary code and
modify its behavior. Before we can start creating shims, we need to
understand how Python’s import system works and what happens before the
first import.</p>
<h2 id="how-does-pythons-import-work">How does Python’s import
work?</h2>
<p>When a Python program imports a module,
e.g. <code>import mymodule</code>, the following happens:</p>
<ul>
<li>Python checks whether this module has already been imported by
looking in <code>sys.modules</code>. If it has, Python uses the already
imported module.</li>
<li>Python looks for the module in <code>sys.path</code> directories.
<code>sys.path</code> is a list of directories where Python looks for
modules. If the module is found, it is imported and added to
<code>sys.modules</code> as a module object. If not, an error is
raised.</li>
</ul>
<p>Example:</p>
<div class="sourceCode" id="cb1"><pre
class="sourceCode python"><code class="sourceCode python"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="bu">print</span>(<span class="st">"math"</span> <span class="kw">in</span> sys.modules)</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> math</span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="bu">print</span>(<span class="st">"math"</span> <span class="kw">in</span> sys.modules)</span>
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="bu">print</span>(sys.modules[<span class="st">"math"</span>])</span>
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="op">>>></span> <span class="va">False</span></span>
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a><span class="op">>>></span> <span class="va">True</span></span>
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="op">>>></span> <span class="op"><</span>module <span class="st">'math'</span> <span class="im">from</span> <span class="st">'/opt/.../lib/python3.14/lib-dynload/math.cpython-314-darwin.so'</span><span class="op">></span></span></code></pre></div>
<p>So we can see that after the import, the <code>math</code> module is
in <code>sys.modules</code> and we can even see the path to its compiled
<code>.so</code> file.</p>
<h2 id="but-what-happens-even-before-the-first-import">But what happens
even before the first import?</h2>
<p>Now that we know what <code>sys.modules</code> and
<code>sys.path</code> are, we will be able to understand why Python does
the things it does before the first import.</p>
<p>When Python starts, it imports a built-in <code>site</code>
module:</p>
<div class="sourceCode" id="cb2"><pre
class="sourceCode python"><code class="sourceCode python"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> sys</span>
<span id="cb2-2"><a href="#cb2-2" aria-hidden="true" tabindex="-1"></a><span class="bu">print</span>(<span class="st">"site"</span> <span class="kw">in</span> sys.modules)</span>
<span id="cb2-3"><a href="#cb2-3" aria-hidden="true" tabindex="-1"></a><span class="op">>>></span> <span class="va">True</span></span></code></pre></div>
<p>Note that we can use Python’s <code>-S</code> flag to avoid importing
<code>site</code>:</p>
<div class="sourceCode" id="cb3"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="ex">venv/bin/python</span> <span class="at">-S</span> <span class="at">-c</span> <span class="st">"import sys; print('site' in sys.modules)"</span></span>
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a><span class="op">>>></span> False</span>
<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a><span class="ex">venv/bin/python</span> <span class="at">-c</span> <span class="st">"import sys; print('site' in sys.modules)"</span></span>
<span id="cb3-4"><a href="#cb3-4" aria-hidden="true" tabindex="-1"></a><span class="op">>>></span> True</span></code></pre></div>
<p>The role of <code>site</code> is to prepare the Python interpreter.
It:</p>
<ol type="1">
<li><p><strong>Adds <code>site-packages</code> to
<code>sys.path</code>.</strong></p>
<p><code>sys.path</code> is a list of directories where Python looks for
modules. <code>site-packages</code> is where third-party (not standard
library) packages are installed.</p>
<div class="sourceCode" id="cb4"><pre
class="sourceCode python"><code class="sourceCode python"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a>[ </span>
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a> <span class="co"># current directory (modules in our project).</span></span>
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a> <span class="st">''</span>,</span>
<span id="cb4-4"><a href="#cb4-4" aria-hidden="true" tabindex="-1"></a> <span class="co"># standard library from a zip</span></span>
<span id="cb4-5"><a href="#cb4-5" aria-hidden="true" tabindex="-1"></a> <span class="st">'/opt/homebrew/Cellar/python@3.14/.../python314.zip'</span>,</span>
<span id="cb4-6"><a href="#cb4-6" aria-hidden="true" tabindex="-1"></a> <span class="co"># standard library</span></span>
<span id="cb4-7"><a href="#cb4-7" aria-hidden="true" tabindex="-1"></a> <span class="st">'/opt/homebrew/Cellar/python@3.14/.../lib/python3.14'</span>,</span>
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a> <span class="co"># compiled code (.so)</span></span>
<span id="cb4-9"><a href="#cb4-9" aria-hidden="true" tabindex="-1"></a> <span class="st">'/opt/homebrew/Cellar/python@3.14/.../lib/python3.14/lib-dynload'</span>,</span>
<span id="cb4-10"><a href="#cb4-10" aria-hidden="true" tabindex="-1"></a> <span class="co"># site-packages !!</span></span>
<span id="cb4-11"><a href="#cb4-11" aria-hidden="true" tabindex="-1"></a> <span class="st">'/Users/youssefboughizane/../venv/lib/python3.14/site-packages'</span></span>
<span id="cb4-12"><a href="#cb4-12" aria-hidden="true" tabindex="-1"></a>]</span></code></pre></div>
<p>If we run the previous command with <code>-S</code> (without
<code>site</code>), we will not see the last line. Don’t trust me, run
it!</p></li>
<li><p><strong>Processes <code>.pth</code> files in
<code>site-packages</code> directories.</strong></p>
<p><code>.pth</code> files contain paths to directories that we want in
<code>sys.path</code>. This can, for example, be used to import a
library. If you have in <code>site-packages</code> a
<code>mypackage-includer.pth</code> containing
<code>/opt/mypackage/lib/python</code>, then this path will be added to
<code>sys.path</code> and <code>import mypackage</code> will be
possible. Why not install the package directly in
<code>site-packages</code>? Good question, I am not sure about the
answer, but I guess not everyone wants to use a package manager,
especially in the early days. Nowadays, we can use these
<code>.pth</code> files to inject some interesting code in our apps
;)</p>
<p>Experiment: I will create a package <code>mypackage</code> outside
our <code>sys.path</code>. I will create it in the parent dir of the
current project dir with</p>
<div class="sourceCode" id="cb5"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="bu">echo</span> <span class="st">'print("Hi from mymodule")'</span> <span class="op">></span> ../mymodule.py</span></code></pre></div>
<p>Let’s try to import it:</p>
<div class="sourceCode" id="cb6"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="ex">venv/bin/python</span> <span class="at">-c</span> <span class="st">"import mymodule"</span></span>
<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a><span class="op">>>></span> Traceback <span class="kw">(</span><span class="ex">most</span> recent call last<span class="kw">)</span><span class="bu">:</span></span>
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a><span class="op">>>></span> File <span class="st">"<string>"</span><span class="ex">,</span> line 1, in <span class="op"><</span>module<span class="op">></span></span>
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a><span class="op">>>></span> import <span class="ex">mymodule</span></span>
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a><span class="op">>>></span> ModuleNotFoundError: <span class="ex">No</span> module named <span class="st">'mymodule'</span></span></code></pre></div>
<p>Fails as expected. Now let’s put a <code>.pth</code> file that points
to it in <code>site-packages</code>. First, we create the
<code>.pth</code> with the path of our parent
<code>$(pwd)/..</code>.</p>
<div class="sourceCode" id="cb7"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="bu">echo</span> <span class="st">"</span><span class="va">$(</span><span class="bu">pwd</span><span class="va">)</span><span class="st">/.."</span> <span class="op">></span> venv/lib/python3.14/site-packages/mymodule-includer.pth</span></code></pre></div>
<p>Now we can test again.</p>
<div class="sourceCode" id="cb8"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="ex">venv/bin/python</span> <span class="at">-c</span> <span class="st">"import mymodule"</span> </span>
<span id="cb8-2"><a href="#cb8-2" aria-hidden="true" tabindex="-1"></a><span class="ex">Hi</span> from mymodule</span></code></pre></div>
<p>Amazing! This means we have at least a partially correct
understanding of this. Note that <code>.pth</code> files can also
contain code, we will explore this later.</p></li>
<li><p><strong>Imports <code>sitecustomize</code> from
<code>sys.path</code> directories.</strong></p>
<p>The <code>sitecustomize.py</code> file is here to let you customize
your Python. At startup, Python will do
<code>import sitecustomize</code> which loops through
<code>sys.path</code> in order and imports the first occurrence of
<code>sitecustomize</code> it finds. Let’s test this! Let’s create a
<code>sitecustomize.py</code> in the current dir.</p>
<div class="sourceCode" id="cb9"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="bu">echo</span> <span class="st">'print("Hi from custom sitecustomize.py")'</span> <span class="op">></span> sitecustomize.py</span></code></pre></div>
<p>Let’s launch Python code that just imports <code>sys</code> and that
has the current dir prepended to <code>sys.path</code>.</p>
<div class="sourceCode" id="cb10"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb10-1"><a href="#cb10-1" aria-hidden="true" tabindex="-1"></a><span class="va">PYTHONPATH</span><span class="op">=</span><span class="va">$PWD</span> <span class="ex">venv/bin/python</span> <span class="at">-c</span> <span class="st">"import sys"</span></span>
<span id="cb10-2"><a href="#cb10-2" aria-hidden="true" tabindex="-1"></a><span class="ex">Hi</span> from custom sitecustomize.py</span></code></pre></div>
<p>Fantastic.</p></li>
</ol>
<h2 id="lets-have-fun-intercepting-print">Let’s have fun intercepting
<code>print</code></h2>
<p>Enough theory, let’s start living. We saw that:</p>
<ul>
<li>Python automatically imports a module <code>site</code></li>
<li><code>site</code> executes all the <code>.pth</code> files in
<code>site-packages</code>.</li>
<li><code>.pth</code> files can contain code. In particular, they can
import a module that we create.</li>
<li>If this module is in <code>sys.path</code>, it will be imported and
executed.</li>
</ul>
<p>With these observations, we can cook up a way to intercept Python’s
<code>print</code> function. We will create a function that intercepts
<code>print</code>, we will import its module in a <code>.pth</code>
file, and we will put that <code>.pth</code> file in
<code>site-packages</code>. This way, when Python starts, it will import
our module and our function will intercept <code>print</code>.</p>
<p>So here’s our <code>my_print_hook.py</code> module that will
intercept <code>print</code>.</p>
<div class="sourceCode" id="cb11"><pre
class="sourceCode python"><code class="sourceCode python"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> builtins</span>
<span id="cb11-2"><a href="#cb11-2" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb11-3"><a href="#cb11-3" aria-hidden="true" tabindex="-1"></a>_original_print <span class="op">=</span> builtins.<span class="bu">print</span></span>
<span id="cb11-4"><a href="#cb11-4" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb11-5"><a href="#cb11-5" aria-hidden="true" tabindex="-1"></a><span class="kw">def</span> intercepted_print(<span class="op">*</span>args, <span class="op">**</span>kwargs):</span>
<span id="cb11-6"><a href="#cb11-6" aria-hidden="true" tabindex="-1"></a> _original_print(<span class="st">"[INTERCEPTED INSIDE MY PRINT: ]"</span>, <span class="op">*</span>args, <span class="op">**</span>kwargs)</span>
<span id="cb11-7"><a href="#cb11-7" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb11-8"><a href="#cb11-8" aria-hidden="true" tabindex="-1"></a>builtins.<span class="bu">print</span> <span class="op">=</span> intercepted_print</span></code></pre></div>
<!-- For that we will create a fake print function in `venv/lib/python3.14/site-packages/my_print_hook.py`. -->
<p>Now let’s create a <code>.pth</code> file that will import this
module.</p>
<div class="sourceCode" id="cb12"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb12-1"><a href="#cb12-1" aria-hidden="true" tabindex="-1"></a><span class="bu">echo</span> <span class="st">"import my_print_hook"</span> <span class="op">></span> <span class="st">"</span><span class="va">$SITE</span><span class="st">/my_print_hook.pth"</span></span></code></pre></div>
<p>where <code>$SITE</code> is the path to
<code>site-packages</code>.</p>
<p>Now, the <code>.pth</code> will try to import
<code>my_print_hook</code> but it will not find it in
<code>sys.path</code>. We can fix that with <code>PYTHONPATH</code>:
when we prefix a command with <code>PYTHONPATH=<path></code>,
Python will prepend <code><path></code> to
<code>sys.path</code>.</p>
<p>Let’s test it!</p>
<div class="sourceCode" id="cb13"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb13-1"><a href="#cb13-1" aria-hidden="true" tabindex="-1"></a><span class="ex">venv/bin/python</span> <span class="at">-c</span> <span class="st">"print('hello')"</span></span>
<span id="cb13-2"><a href="#cb13-2" aria-hidden="true" tabindex="-1"></a><span class="op">>>></span> hello</span>
<span id="cb13-3"><a href="#cb13-3" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb13-4"><a href="#cb13-4" aria-hidden="true" tabindex="-1"></a><span class="va">PYTHONPATH</span><span class="op">=</span><span class="va">$(</span><span class="bu">pwd</span><span class="va">)</span> <span class="ex">venv/bin/python</span> <span class="at">-c</span> <span class="st">"print('hello')"</span></span>
<span id="cb13-5"><a href="#cb13-5" aria-hidden="true" tabindex="-1"></a><span class="op">>>></span> <span class="ex">[INTERCEPTED</span> INSIDE MY PRINT WITH PYTHONPATH: ] hello</span></code></pre></div>
<p>Ouh la la, we have successfully intercepted the print function.</p>
<p>In this case, only Python processes that have or inherit
<code>PYTHONPATH</code> set to the current dir will have the intercepted
<code>print</code>.</p>
<p><em>What if we want all the processes to have the intercepted
<code>print</code>?</em></p>
<p>We can put <code>my_print_hook.py</code> in
<code>site-packages</code> and it will be imported by all processes.
Let’s try that: we’ll move our local <code>my_print_hook.py</code> to
<code>site-packages</code> and change it to print a different
message.</p>
<div class="sourceCode" id="cb14"><pre
class="sourceCode python"><code class="sourceCode python"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> builtins</span>
<span id="cb14-2"><a href="#cb14-2" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb14-3"><a href="#cb14-3" aria-hidden="true" tabindex="-1"></a>_original_print <span class="op">=</span> builtins.<span class="bu">print</span></span>
<span id="cb14-4"><a href="#cb14-4" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb14-5"><a href="#cb14-5" aria-hidden="true" tabindex="-1"></a><span class="kw">def</span> intercepted_print(<span class="op">*</span>args, <span class="op">**</span>kwargs):</span>
<span id="cb14-6"><a href="#cb14-6" aria-hidden="true" tabindex="-1"></a> _original_print(<span class="st">"[INTERCEPTED INSIDE MY PRINT WITH SITEPACKAGES: ]"</span>, <span class="op">*</span>args, <span class="op">**</span>kwargs)</span>
<span id="cb14-7"><a href="#cb14-7" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb14-8"><a href="#cb14-8" aria-hidden="true" tabindex="-1"></a>builtins.<span class="bu">print</span> <span class="op">=</span> intercepted_print</span></code></pre></div>
<p>Running it:</p>
<div class="sourceCode" id="cb15"><pre
class="sourceCode python"><code class="sourceCode python"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a>venv<span class="op">/</span><span class="bu">bin</span><span class="op">/</span>python <span class="op">-</span>c <span class="st">"print('hello')"</span> </span>
<span id="cb15-2"><a href="#cb15-2" aria-hidden="true" tabindex="-1"></a><span class="op">>>></span> [INTERCEPTED INSIDE MY PRINT WITH SITEPACKAGES: ] hello</span></code></pre></div>
<p>Voilà, the <code>my_print_hook.pth</code> imports
<code>my_print_hook</code> and, given that <code>site-packages</code> is
in <code>sys.path</code>, the <code>my_print_hook.py</code> is imported
and the <code>print</code> function is intercepted.</p>
<h2 id="conclusion">Conclusion</h2>
<p>I got interested in this because, when working on loading large
models, I was using a custom high-bandwidth program to load the model to
<code>/dev/shm</code> (RAM) and then providing this path to the engine
(sglang). I then wanted to overlap the staging to <code>/dev/shm</code>
with the engine initialization, to hide the staging cost. For this I
needed a barrier between the two phases: if the staging was not
finished, the engine would wait. The issue is that this requires us to
change sglang’s code to add a barrier, which is inconvenient. Using what
we learned today, we can intercept <code>safetensors.safe_open</code>,
add a barrier there that, when satisfied, will let the engine continue.
This is a very powerful technique that can be used to modify the
behavior of existing code without changing it.</p>
</body>
</html>