Nate Tracy-Amoroso’s Homepage

Setting up llvm for inkwell on macOS

2022-05-28T03:00:00+00:00

git clone https://github.com/llvm/llvm-project.git
cd llvm-project
git checkout remotes/origin/release/13.x
mkdir build && cd build
cmake ../llvm -DCMAKE_INSTALL_PREFIX=$HOME/llvm-13
cmake --build . --target install

I’m compiling LLVM for use with inkwell which relies on the LLVM_SYS_130_PREFIX env variable to find llvm (for llvm 13 of course, it’s different for other versions more here).

After installing modify your shell to set this variable to $HOME/llvm-13 or wherever you want llvm (modify the cmake DCMAKE_INSTALL_PREFIX flag as well).

If you want a version other than 13 run git branch -a for a list of remote branches.

You can also find official instructions here: https://llvm.org/docs/CMake.html

What to do when mysql’s .binlog files are filling up the disk

2022-05-28T03:00:00+00:00

Woke up to alerts that websites were down. Quickly find that the disk is full with df -h. Next up why. Install ncdu with apt-get install ncdu and view a tree of my directories and their sizes.

--- / --------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1 GiB [##########] /var
2 GiB [######    ] /usr
6 GiB [##        ] /home
2 GiB [##        ] /snap
1 MiB [          ] /boot
8 MiB [          ] /root
1 MiB [          ] /etc

Dig around in there until I find the binlogs. Okay how to remove them, quick google turns up this to make them automatically expire

SET PERSIST binlog_expire_logs_seconds = 259200;
SET GLOBAL binlog_expire_logs_seconds = 259200;
SHOW BINARY LOGS;

+---------------+-----------+-----------+
| Log_name      | File_size | Encrypted |
+---------------+-----------+-----------+
| binlog.001604 | 104861415 | No        |
| binlog.001605 |   2133604 | No        |
| binlog.001606 |  10123192 | No        |
| binlog.001607 |       157 | No        |
| binlog.001608 |       157 | No        |
| binlog.001609 |  54486285 | No        |
| binlog.001610 |    181294 | No        |
| binlog.001611 |    301159 | No        |
| binlog.001612 |    120357 | No        |
| binlog.001613 |       157 | No        |
| binlog.001614 |  20589546 | No        |
| binlog.001615 |  72575731 | No        |
| binlog.001616 |  41724083 | No        |
| binlog.001617 |       157 | No        |
...

Oh that didn’t clean the old ones. Finally cleared with

PURGE BINARY LOGS TO 'binlog.last_number_i_saw';

Restart a few things and we’re rolling again.

Don’t make me wait

2021-11-08T03:00:00+00:00

Servers are lovely. They run 24/7, they do their work immediately, and they usually manage to do a few things at time. Beautiful.

Enters the engineer.

Suddenly it’s important that we batch process. For “efficiency”. Now our systems sit idle for 59 minutes an hour. Amazing.

There I am $14 spent at one of the largest most advanced tech companies in the world for a copy of a piece of software. A binary.

“It may take up to one hour for your purchase to appear in your digital blah blah blah” -Bad Website

What could they be doing. Credit cards take seconds not minutes to process. Even if they want to give me a unique binary to track pirating that would only take seconds. Alas I shall never know. But hopefully by the time this post is live my download will be ready…

PLEASE stop slowing down computers with human scale constraints. A computer scale batch (for a web server at least) would be in 50ms-1000ms range. If you ever find yourself using hours in computer systems there’s a good chance you’ve fucked things up.

P.S. Am I impatient? Yes. Is $14 a lot No. Is there any reason this shouldn’t be instant NO!

Ask not if async/await is good enough for you, ask if you are you good enough for async/await

2021-09-05T03:00:00+00:00

async Task<int> doesYourCodeLookLikeThisAsync() {
    var ifItDoes = await this.YouMightFindThat();
    var thereWasNoPoint = await to.usingAsyncAwait();
    because.There = await isNoParalellism(that, we, can exploit, here);
    becuase.What = await youActuallyWanted(was, to, make, single-thread, code, faster);
    // but that is hard so instead just use 
    // async await and that'll make everythign faster somehow. 
}

If you actually want to use async/await to make thing faster you need to notice what work can be done in parallel and actually do it in parallel and then await strategically when you need the result.

Here’s what I see a lot. Code that boils down to be blocking non-async code.

var c_data = await this.loadCustomerData();
await this.process(c_data); // wait for customer work to be done
var v_data = await this.loadVendorData(); // then do vendor work
await this.process(v_data);

If you actually want a speedup you need something like this.

var customerWork = Task.Run(async () => {
    var c_data = await this.loadCustomerData();
    await this.process(c_data);
});
var vendorWork = Task.Run(async () => {
    var v_data = await this.loadVendorData();
    await this.process(v_data);
});
Task.WhenAll(customerWork, vendorWork); // wait for both to complete in parallel

This isn’t simpler. It’s quite a bit more complicated. It’s also harder to read. In general I’d only want to implement this kind of solution when perf matters OR if the team is ready to normalize async/await complexity.

If you don’t have perf problems and don’t want to run into async await complexity, code like it’s 1990 (okay this is c# so 2000).

var c_data = this.loadCustomerData();
this.process(c_data);
var v_data = this.loadVendorData();
this.process(v_data);

Scancodes for the windows keyboard in a machine readable format

2021-09-04T03:00:00+00:00

scancodes.txt

00 NOTHING
20 D
40 F6
60 Ctrl F3
80 Alt 9
A0 Alt Dn Arrow
01 ESC
21 F
41 F7
61 Ctrl F4
81 Alt 0
A1 Alt PgDn
02 1
22 G
42 F8
62 Ctrl F5
82 Alt -
A2 Alt Ins
03 2
23 H
43 F9
63 Ctrl F6
82 Alt =
A3 Alt Del
04 3
24 J
44 F10
64 Ctrl F7
84 Ctrl PgUp
A4 Alt / (num)
05 4
25 K
45 Num Lk
65 Ctrl F8
85 F11
A5 Alt Tab
06 5
26 L
46 Scrl Lk
66 Ctrl F9
86 F12
A6 Alt Enter (Num)
07 6
27 ; :
47 Home
67 Ctrl F10
87 Shift F11
08 7
28 ' "
48 Up Arrow
68 Alt F1
88 Shift F12
09 8
29 ` ~
49 Pg Up
69 Alt F2
89 Ctrl F11
0A 9
2A Left Shift
4A - (num)
6A Alt F3
8A Ctrl F12
0B 0
2B \ |
4B 4 Left Arrow
6B Alt F4
8B Alt F11
0C - _
2C Z
4C 5 (num)
6C Alt F5
8C Alt F12
0D = +
2D X
4D 6 Rt Arrow
6D Alt F6
8C Ctrl Up Arrow
0E BKSP
0F Tab
10 Q
11 W
12 E
13 R
14 T
15 Y
16 U
17 I
18 O
19 P
1A [ {
1B ] }
1C Enter
1D Ctrl
1E A
1F S
2E C
2F V
30 B
31 N
32 M
33 , <
34 . >
35 / ?
36 Right Shift
37 Print Screen
38 Alt
39 Space
3A CapsLock
3B F1
3C F2
3D F3
3E F4
3F F5
4E + (num)
4F 1 End
50 2 Dn Arrow
51 3 Pg Dn
52 0 Ins
53 Del .
54 Shift F1
55 Shift F2
56 Shift F3
57 Shift F4
58 Shift F5
59 Shift F6
5A Shift F7
5B Shift F8
5C Shift F9
5D Shift F10
5E Ctrl F1
5F Ctrl F2
6E Alt F7
6F Alt F8
70 Alt F9
71 Alt F10
72 Ctrl PtScr
73 Ctrl L Arrow
74 Ctrl R Arrow
75 Ctrl End
76 Ctrl PgDn
77 Ctrl Home
78 Alt 1
79 Alt 2
7A Alt 3
7B Alt 4
7C Alt 5
7D Alt 6
7E Alt 7
7F Alt 8
8E Ctrl - (num)
8F Ctrl 5 (num)
90 Ctrl + (num)
91 Ctrl Dn Arrow
92 Ctrl Ins
93 Ctrl Del
94 Ctrl Tab
95 Ctrl / (num)
96 Ctrl * (num)
97 Alt Home
98 Alt Up Arrow
99 Alt PgUp
9A <none>
9B Alt Left Arrow
9C <none>
9D Alt Rt Arrow
9E <none>
9F Alt End

The Permissions-Policy header still sucks; it’s been 3 years

2021-08-23T02:00:00+00:00

The HTTP header permissions policy (formerly feature-policy) controls what features the page can use. Eg. USB, accelerometer, battery, camera…

The most secure permissions policy header looks like this (apache config):

Header always set Permissions-Policy "accelerometer=(), ambient-light-sensor=(), autoplay=(), battery=(), camera=(), display-capture=(), document-domain=(), encrypted-media=(), execution-while-not-rendered=(), execution-while-out-of-viewport=(), fullscreen=(), gamepad=(), geolocation=(), gyroscope=(), layout-animations=(), legacy-image-formats=(), magnetometer=(), microphone=(), midi=(), navigation-override=(), oversized-images=(), payment=(), picture-in-picture=(), publickey-credentials-get=(), speaker-selection=(), sync-xhr=(), unoptimized-images=(), unsized-media=(), usb=(), vibrate=(), vr=(), screen-wake-lock=(), web-share=(), xr-spatial-tracking=()"

Yuck! Those =() are empty lists meaning nothing is allowed to use the feature

The worst part is this example will be obsolete once google shoves a new feature through w3c and we all need to disable the new intra-ocular-laser feature.

Please give us a way to specify that the default is to deny the use of a feature.

Permissions-Policy: default()

Lo and behold my idea isn’t original. Someone suggested the same thing 3 years ago. I have no idea how these things are standardized but maybe upvoting the first post here will help:

github.com/w3c/webappsec-permissions-policy/issues/189

There are lots of more complicated suggestions the comments. I don’t personally see a need for them. A good default of DENY and a whitelist of features you’re actually using seems like a fine approach.

Permissions-Policy: default=(),accelerometer('self')

Feel free to shoot me an email if you think I’m wrong.

PS:

Yes Yes I know this is small potatoes and only even comes into play once someone manages to get malicious js running on your page. But still, the fix is easy, let’s do it.

What’s up with null in c# (and c, cpp, etc…)

2021-08-22T03:00:00+00:00

I constantly see this at work

public class Tmp {
	public void DoSomething(TheType a, TheOtherType b, AnOtherType c) {
		if (a == null) {
		    throw new ArgumentNullException("a");
		}
		if (b == null) {
		    throw new ArgumentNullException("b");
		}
		if (c == null) {
		    throw new ArgumentNullException("c");
		}
		Console.WriteLine("Done");
	}
}

And I die a little inside every time.

Didn’t I say I wanted a TheType? Why is it possible that I am instead given a bomb type instead that’ll explode if I touch it. Eg:

public class TheType {}
public class TheOtherType {}
public class AnotherType {}

public class Program
{
	public static void Main()
	{
		var tmp = new Tmp();
		tmp.DoSomething(null, null, null); // Somehow null is a valid `TheType`
	}
	
}
public class Tmp {
	public void DoSomething(TheType a, TheOtherType b, AnotherType c) {
		a.ToString();
		Console.WriteLine("Done");
	}
}

And BOOM, exception.

Run-time exception (line 19): Object reference not set to an instance of an object.

Stack Trace:

[System.NullReferenceException: Object reference not set to an instance of an object.]
   at Tmp.DoSomething(TheType a, TheOtherType b, AnotherType c) :line 19
   at Program.Main() :line 13

My poor function DoSomething asked for a TheType was handled literally nothing and the type system just let it happen.

This is made worse by the fact that c# has the idea of nullability and they get it right for primitives. Here’s a program that won’t compile. (Wouldn’t it be great if the earlier examples didn’t compile too!)

public class Program
{
	public static void Main()
	{
		long? a = 123;
		var mapping = new Dictionary<long,long>();
		mapping[a] = 1;
	}
}

Error(s):
(15:11) Argument 1: cannot convert from 'long?' to 'long'

Hey look at that it noticed that a nullable long long? is not the same as a long.

And C# even has this pretty syntax:

public class Program
{
    public static void M`ai``n()
    {
        long? a = 123;
        if (a is long notNullA) {
            var mapping = new Dictionary<long,long>();
            mapping[notNullA] = 1;
        }
    }
}

We can unwrap our long? into a long and get a new variable with the correct type.

Of course you can do the same unwrapping trick for classes but you shouldn’t need to double check every arguement is the type you asked for. Classes should default to non-nullable just like the primitives.

The current behavior is un-intuitive, boilerplate heavy, and inflexible.

mysql 5.7 per query timeout with optimizer hints in rails 6.0 and plain sql

2021-07-19T03:00:00+00:00

MySQL >= 5.7 allows a per query timeout as an optimizer hint. It only works for SELECT.

Here’s a 500ms timeout:

SELECT /*+ MAX_EXECUTION_TIME(500) */ COUNT(DISTINCT model.id)
FROM `model`

And in rails you can do

begin
  count = Model.all.optimizer_hints("MAX_EXECUTION_TIME(500)").count
rescue ActiveRecord::StatementTimeout
  puts "Uh-oH!"
end

It’s been easy to set a GLOBAL max query execution time in mysql for awhile. However this breaks mysqldump and other long running commands. post. I prefer an opt-in system where I can wrap only the problematic queries.

So I’ve been searching for a way to do better per-query timeouts and here it is!

MySQL docs

Struggling to replace master slave, how about shepard sheep?

2021-07-01T03:00:00+00:00

Master and slave are everywhere in computer science.

A master database server handles the writes and passes them out to read replicas or slaves.

The git master branch.

To name the biggest two. For some these words conjure bad imagery and worse history. There have been lots of attemps to replace them. There’s also been a some maligning the change as stupid, ageist, and other nonesense mostly bad arguements on quora

There have been a lot of a suggestions

parent/child
leader/follower
primary/replica

But all of these share one thing. They lack

IMAGINATION

So I present my own alternative.

source

Shepard and Sheep

Pros:

Contains sheep, everyone loves sheep
Not controversial
Contains shepard, good old timey imagery
Fun to say shhhh-eppp-ard shhhh-eeeyyypp

Cons:

¿ Doesn’t include any other animals ?

Please help me spread the word. A suitable replacement has been found. It’s time to

git pull origin shepard
kubectl create -f sheep

Why is there a chip shortage? Crap like this!

2021-05-23T03:00:00+00:00

We’ve heard a lot about how a lack of silicon chips is crippling manufacturing. I hold the opposite view, manufacturing is being crippled by a surplus of silicon. I just left the lovely state of Florida and while there I saw this fantastic light switch panel.

It has 3 parts

A normal lightswitch (it controls nothing)
A lovely fanimation ceiling fan that actually controls the lights as well as the fan
A normal lightswitch (it also does nothing)

While I’m sure these people make great fans they’ve massively overcomplicated it in a few ways. First there’s an on off switch that control the PANEL not the fan. You can slide that to off and the fan stays on but the buttons now do nothing. Why does the panel itself have state that is separate from the lights/fan state?

Second they’ve replaced lovely rocking light switches which are easy and intuitive to use and replaced them with a small green button. Switches are great they have two states just like lights on and off vs up and down. Buttons have no state.

Third they violate assumptions; we expect to control fan speed either not at all or by turning a nob. They use neither and have a series of buttons.

Had they needed to make this product in 1950 it would have been two light switches. That’s it. No circuit board behind the lightswitch, no buttons, nothing. Instead they use more tech than they need by about 70 years.

A few other things I’d like to see have their circuit boards ripped out:

Microwaves (just 1 button to add 30s please, idk what the rest are for)
Cars (not all the circuit boards, just no over the air updates please)
Smart-insert-noun (lightblubs, refrigerators, ac units, etc.)

Anyway next time you buy something choose the dumb version. It’ll last twice as long and cost half as much.