Newsfeed

Meanwhile, elsewhere…

  • Mutually assured intel

    Bruce Schneier:

    Supply-chain security is an incredibly complex problem. [National]-only design and manufacturing isn’t an option; the tech world is far too internationally interdependent for that.

    We can’t trust anyone, yet we have no choice but to trust everyone.

Doubly detached IRC

Thursday, 8 Feb 2018

For long-running terminal sessions such as IRC, I’ve long used dtach (or Screen, or tmux, it doesn’t matter) to be able to leave them running on a server without having to be connected to it… just like everyone else does. But recently I put together some simple facts in a retrospectively obvious way that I haven’t heard of anyone else doing, and thus achieved a little quality of life improvement.

I wrote before about how much I love Mosh when I need it. And when I had the insight I am writing about, I was in a situation where I seriously needed it. (If you don’t know what Mosh is, read about it first, because ⓐ you’re missing out and ⓑ the rest of this article won’t make much sense otherwise.)

Here’s the thing about Mosh, though: it still has to go through regular SSH in order to bring up a Mosh session. And on a severely packet-lossy connection, that alone can be hell.

The real beauty of Mosh comes to the fore only if you keep the session around once it’s set up. As long as it’s up, then no matter how bad the packet loss, you get decent interactivity. But to fully benefit from that, you have to avoid tearing the session down.

My problem is that try as I might, I have never been able to break with my compulsion to close terminal windows once I am done with them. For IRC that means sooner or later I’ll want to detach from a session I’m not actively chatting in. And because I use Mosh to run dtach on the remote end, detaching from IRC means that the dtach client exits on the remote end… which tears down the Mosh session.

The simple fact that suddenly occurred to me is that I can also use dtach on my end of the connection, in front of Mosh:

dtach -A ~/.dtach/irssi mosh hostname dtach -A ~/.dtach/irssi irssi

Now when I detach, it is only from my local dtach session, not the one on the server. So the Mosh session behind it sticks around – without me having to keep the terminal window open.

The upshot is a dtach ↔ Mosh ↔ dtach sandwich which gives me the full benefits of Mosh.


If you want to use this yourself, note the procedure to end the Mosh session in this case:

dtach -a ~/.dtach/irssi -E # and then press the detach shortcut

The -E switch disables the keyboard shortcut for detaching in the local dtach client. This means when you press the shortcut it gets sent to the remote dtach client. What follows then is exactly the same chain of events as always when you detach from the remote dtach session: the remote dtach client exits, so Mosh exits. And therefore the local dtach session ends as well. Detaching from the remote end thus brings the whole edifice down, and the simplest way to trigger that is to disable detaching on the local end.

Backoff

Friday, 15 Sep 2017 [Tuesday, 19 Sep 2017]

Some time ago I had occasion to implement (mostly exponential) back-off in an application for the first time. This is not a hard problem, but at the outset I expected it to be one of those annoying cases where the code is only clear to read when you are immersed in the problem it solves.

Not so. It turns out there is a trivially simple algorithm if you pick the right form of stored state – namely, a pair of timestamps: (last_success, next_retry). The essential form of the algorithm goes like this:

if succeeded { last_success = now }
next_retry = last_success, i = 0
until next_retry > now {
  next_retry += step_size(i)
  i += 1
}

Because this recalculates the scheduling for the next retry by starting over from the previous success, every time, it is totally resilient against all fluctuations in the environment. Belated or missing retry attempts have no effect on its output. Even swapping the step_size function for an entirely different one mid-flight just works!

At the same time, it is trivial to reason out and verify that this algorithm works correctly.

I was quite pleased.


(In practice you will likely not have a separate step_size function and i counter but rather some kind of step variable iterated along with next_retry. But here I wanted to abstract away from the specific formula used.)

Update: As prompted by a question from Matthew Persico, let me clarify that my use case is scheduling polls that succeed only intermittently, meaning that I always want to wait at least once between attempts, which is why I used “until next_retry > now”.

If instead you want to add backoff to an operation that only fails intermittently (e.g. draining a buffer to I/O) then you’ll want to use “while next_retry < now” for your loop, so you can have zero-delay back-to-back attempts.

Fixing a Google Chrome failure to save passwords

Sunday, 2 Jul 2017

This is a kind of post that people used to write back in the heady early days of blogging and a more communal web: putting something out there to help Google help other people.

The problem

For some time I had been having an irritating persistent failure with Google Chrome that I could not find an answer for:

  • After logging into some website, it would offer me to save the password, as usual.
  • I would click on the save button.
  • Chrome would not show any kind of error.
  • But the password would not be saved:
  • It was not filled in automatically next time I went to the same site.
  • No password at all showed up in the list on chrome://settings/passwords – the list just stayed blank no matter what I did.

Mysteriously, a handful of passwords did get stored, somehow, somewhere. Chrome could fill those in even as it woudn’t list them on the settings screen. I checked the MacOS keychain and did not find them there, so they had to be stored by Chrome, even though it refused to show me them.

The quest

Searching the web about my problems, almost all answers I could find related to the case of people who are logged into Google within Chrome and use its password syncing service… which I don’t. I simply want my passwords saved locally.

The few answers I did find that seemed to relate to my situation invariably suggested resetting one’s profile. Now, that approach does appear not to be mere superstition: of the people I found who had this problem, the ones who reported back all wrote that resetting their profile fixed the problem. So I had a way of making the problem go away – but I also have a lot of data in my profiles. It’s not just my bookmarks. I have tweaked many of the settings, individually for each profile (the whole point of using profiles, after all), and I also use a number of extensions, many of which themselves have extensive configurations. Recreating that all is a big task.

I want password auto-fill fixed while keeping my profiles intact. I am only willing to lose my stored passwords. (I save them in a password manager first.)

So I went poking around in the directories where Chrome stores its profiles and other user data. There’s no need to look far: there’s a file called Login Data. This is an SQLite database (like most of Chrome’s user data files). It can be opened using the sqlite3 command line utility and examined using SQL queries. I did that, and there they were, my mysteriously saved passwords… plus a bunch more.

I also discovered that some of the data in those tables is scrambled in some form. Presumably there are several separatedly stored pieces of data required to unscramble that data, and some of those pieces somehow become mismatched on my system – I don’t know how nor why, and was too lazy to research. All I cared was that this looked like the right vicinity.

As an experiment, I moved the Login Data file and its Login Data-journal pair out of one profile… and bingo, password auto-fill started working there as expected. After saving a password, Chrome would subsequently successfully auto-fill it as well as list it on the saved passwords screen.

Good enough for me.

The solution

Deleting the files Login Data and Login Data-journal from a profile fixes password saving in that profile – without affecting any other data in it. A full profile reset is not necessary – you can reset just the password storage by deleting just the files that it uses.

This does mean you lose any passwords you had stored previously, unfortunately. But since you cannot really access them any more anyway, that data loss has effectively already happened by the time you delete the files.

Instructions

  1. Quit Chrome.

  2. Go to the directory where Chrome stores its user-specific data, below your user home directory:

    Mac
    ~/Library/Application Support/Google/Chrome
    Linux
    ~/.config/google-chrome
    Windows
    %UserProfile%\AppData\Local\Google\Chrome\User Data
  3. From there, go into the directory called Default if you want to fix your main profile, or into Profile 1 or Profile 2 etc. to fix one of your extra profiles.

  4. Delete the files Login Data and Login Data-journal.

  5. Repeat for other profiles as necessary.