tech stuff.

Archive for the ‘Internet’ Category

Migrate ProtonMail to GMail

leave a comment »

After trying ProtonMail for a year, I decided to migrate my email domain back to Google Workspace from ProtonMail. I like Proton, but in the end my users weren’t happy with the available features, and I was more interested in keeping my users happy.

Anyway, I needed to migrate a corpus of email from ProtonMail to GMail, and all the Internet had to offer was that I should use the Import-Export app to export my mail to mbox, use an extension to load the mbox into Thunderbird, and copy the mail using GMail IMAP.

GMail is great at handling millions of users simultaneously accessing their email, but high throughput from a single user isn’t its strong suit. When I attempted to copy more than 10 messages at a time via IMAP the operation would usually time out.

Here’s a solution for copying email from ProtonMail to GMail that worked better than IMAP for me (for Workspace users this requires imports from webmail hosts to be on):

  1. Export ProtonMail to a local mbox file using Proton’s Import-Export app.
  2. Copy the mbox file to a short-lived Cloud VM running a POP3 server.
  3. Use GMail’s import mail feature to have GMail fetch the messages from your server.

This worked really well. GMail connected every few minutes and pulled 200 messages at a time until all messages were transferred.

Creating a temporary mail server was easier than it sounds. The rough steps I took were:

  1. Create a GCE Instance running Debian 10. (I used a C3 instance because it was in free public preview, but GMail is conservative in its fetching so I would use an e2-micro next time.)
  2. Add GCE firewall rules to allow traffic to ports 80 (for Lets Encrypt) and 995 (for POP3 with SSL).
  3. Assign a DNS record for the host and follow the certbot instructions to obtain a Lets Encrypt certificate.
  4. Install dovecot-pop3d, updating the config for the path to the Lets Encrypt SSL certificate and setting the mail_location to ~/mbox.
  5. Add a mail user.

Almost surprisingly, that’s all I had to do. The whole process, including setting up the mail import, took around an hour.

Written by Lee Verberne

2023-04-15 at 18:28

Posted in Internet

Hardening SSH

leave a comment »

Here’s an interesting article on Secure Secure Shell.

Written by Lee Verberne

2015-01-06 at 21:50

Posted in Internet

Tagged with

Backup to Google Cloud Storage using duplicity 0.6.22

with 2 comments

My patch to add support to duplicity  for Google Cloud Storage was merged and released with duplicity version 0.6.22.  Now backing up to GCS is as easy as backing up to S3.  Here are the steps:

  1. Install duplicity >= 0.6.22.
  2. Enable Interoperable Access in the Cloud Storage Dashboard.
  3. Generate Interoperable Storage Access Keys in the Cloud Storage Dashboard.
  4. Create your bucket:
    $ gsutil mb -c DRA gs://BUCKETNAME

    The -c DRA flag enables Durable Reduced Availability for this bucket, which makes sense for backups.

  5. Run gsutil config -a to generate a ~/.boto configuration file with your key info.  Alternatively (or if you don’t use gsutil) you can set the GS_ACCESS_KEY_ID and GS_SECRET_ACCESS_KEY environment variables.
  6. Backup using a gs:// URL.  For example:
    $ duplicity --full-if-older-than 1M --exclude /home/user/.cache \
        /home/user gs://BUCKETNAME/backups/user

Written by Lee Verberne

2013-08-29 at 22:42

Posted in Internet

Tagged with , , ,

Rackspace: meh. not that fanatical.

with one comment

I had such high hopes for Rackspace.  I’ve been hearing so many great things, I couldn’t help but get my hopes up.  My experience with Rackspace was anything but good, however.

It’s not like my request was complicated.  I wanted about 10 servers in managed colocation on a private switch behind a pfsense firewall.  Managed colocation means that I’ll do all the work of configuring them.  All Rackspace has to do is stick some servers in a rack and connect them with cables.  This is part of their advertised service offering.

Talking to the Rackspace sales team was an ordeal.  The problem wasn’t that I had to go through four tiers of sales people — though I did — or that they didn’t get back to me when they said they would — though they didn’t.  The biggest problem I had is that every time I’d advance through another tier of sales people, they would contradict the previous tier.  Every time I talked to someone new they would take something off the table until, in the end, we arrived at a solution that didn’t resemble what I wanted in the first place.

The last straw was when the latest salesperson told me that he just found out about a new rule that all managed colocation customers are required to rent a Cisco ASA firewall.  My BSD firewalls aren’t good enough.  This seems especially confusing since (presumably) I wouldn’t have to administer a Cisco firewall if I were to just rent 10 dedicated servers from them.

Regardless, for $600 per month per server I should be able to configure them in whatever way I want.  That’s way too steep a price to put up with all this run around.

I shall make a point of never doing business with a company who doesn’t list their prices on their website.

Written by Lee Verberne

2010-05-14 at 15:36

Posted in Internet

racoon only matches against the first IP subjectAltName?

with 3 comments

I haven’t examined the source yet to make sure that I’m right, but imperical evidence leads me to believe that racoon is only recognizing the first IP field in an x509v3 subjectAltName extension.  That is, for the following certificate:

X509v3 Subject Alternative Name:
DNS:arthur.example.org, IP Address:192.168.35.24, IP Address:10.14.82.152

It appears that only the 192.168.x.x address will be accepted as a valid ID by racoon.  Requests with an ID of 10.14.82.152 will be discarded with the error message: ID mismatched with subjectAltName.  So far I’ve only tested this with anonymous remote nodes.

Written by Lee Verberne

2009-03-09 at 07:05

Posted in Internet, Unix-type stuff

Tagged with

racoon requires subjectAltName for x509 IKE

leave a comment »

Having trouble getting your ipsec working with x509 certs?  It would appear that racoon requires the subjectAltName extension to be set.  It won’t use the CN.  You have to set a subjectAltName field even if it contains nothing besides a copy of the CN.

Heed this warning, or you’ll fall victim to the following:

racoon: 2008-12-02 14:47:21: ERROR:
racoon: 2008-12-02 14:47:21: ERROR: failed to get subjectAltName
racoon: 2008-12-02 14:47:21: ERROR: no peer's CERT payload found.

Of course… the misery that is tricking openssl to create a cert with the subjectAltName in it is outside the scope of this simple blog entry. Maybe a lengthy one at a later date…

http://www.mail-archive.com/openssl-users@openssl.org/msg47641.html

Written by Lee Verberne

2008-12-02 at 21:12

Posted in Internet, Unix-type stuff

Tagged with

Server load balancing in pfSense 1.3

with 2 comments

I was poking around at the 1.3-ALPHA-ALPHA pfSense, and it looks like slbd is going to be replaced with OpenBSD’s relayd!  I can’t tell you how happy I am about this development.  Not only will relayd increase pfSense’s feature set by some order of magnitude, I have hope that it will make pfSense a viable load balancer.

Though pfSense has load balancing features right now, its limitations make it mostly unusable for my applications.  The way that slbd operates breaks session stickiness.  I’m not sure why, but I suspect that slbd is updating its pf anchor unconditionally per polling period which causes pf to reset its session state.  In addition to breaking stickiness, this causes load to be balanced unfairly.

I’ll have to do some testing to see if relayd has this say behavior.  If nothing else, though, I’m excited about a whole new host of features, some of which I haven’t seen since I last touched a NetScaler:

  • Layer 7 (in addition to Layer 3) load balancing
  • Layer 7 health monitoring (http request/response checking)
  • In-Flight HTTP request/response modifications (such as appending Remote-Host header)
  • Additional balancing algorithms, including one that can be sticky on HTTP headers/cookies

All-in-All pretty fantastic.  I don’t imagine we’ll see 1.3 until Q2 of 2009, though, so it may be worth back-porting it as a package for 1.2.1.

Written by Lee Verberne

2008-09-06 at 21:52

Posted in Internet, pfsense

Tagged with ,

What MAC address should I use for my virtual server?

with 2 comments

I’m creating a new virtual server, and I need a MAC address to assign to it.  (My provisioning/build system requires pre-allocated MACs.)  Instead of pulling one out of thin air, I thought I’d look up what address range I should be using.  It seems surprisingly difficult to find a straight answer on this one, though.

Locally Administered Addresses

It looks like the 802 standard includes provisions for locally administered addresses versus universally administered addresses.  If the second bit transmitted on the wire is a 1, then the address is considered locally administered.  If the second bit is a 0, then address is globally administered and the first 24-bits must be a globally unique OUI assigned by the IEEE.

Note that this is the second bit on the wire.  It would be the second least-significant bit of the most significant byte when represented in hex.  That is, 02:xx:xx:xx:xx:xx is local while 04:xx:xx:xx:xx:xx is universal.

Great, so half of the entire MAC address space is available for me to choose from, right?  I think so, but I haven’t really seen used with virtual servers.

IEEE Private OUI

The IEEE is the authority responsible for keeping the first 24 bits of the address unique.  Vendors apply for an Organizationally Unique Identifier (OUI) which becomes the first 24 bits.  The IEEE has assigned AC-DE-48 to “PRIVATE”.  This looks to be the MAC address equivalent of RFC1918 private IP addresses (10.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12).

Which should I use?

As best as I can tell, locally administered addresses were intended for the purpose of overriding a burned-in/firmware-provided assigned address.  In my case of assigning to a virtual server, I’m trying to emulate an assigned address, not override one.  Using the IEEE private OUI seems like a better fit.

Then I still have the option of overriding my privately unique address with a locally assigned address.  Because that seems like a good idea.  Or something.

Update: Xen Virtuals

It looks like XenSource has registered 00-16-3E with IEEE for use with Xen Virtuals as well.  XenSource recommends using this range.  ymmv.

Written by Lee Verberne

2008-09-05 at 15:23

Posted in Internet

Tagged with , , ,

Free Webmail Username Restrictions

with one comment

What criteria determine a valid username for the top web mail providers?  This seems like it would be useful for filtering that list of a couple hundred thousand e-mail addresses we’ve all got lying around.

Yahoo! Mail Basic Restrictions (yahoo.com, ymail.com, rocketmail.com):

  • 4 to 32 characters
  • Starts with a letter
  • Letters, numbers, underscores and one period (.)
  • No consecutive underscores or period, or combination of the two ([_.]{2}
  • Cannot end with underscore or period

Regex for invalid username:

(?:^.{,3}$|^.{33,}$|^[^a-z]|[^a-z0-9_.]|\..*\.|[_.]{2}|[_.]$)

Gmail Basic Restrictions (gmail.com):

  1. 6 to 30 characters
  2. Only letters, numbers and periods
  3. Starts and ends alphanumeric
  4. No consecutive periods (also required by RFC 2822)

Because of the restriction on the character set (#2), RFC 2822 compliance guarantees #3 & #4.  As such, these won’t appear in the following regex.

Regex for invalid username:

(?:^.{,5}$|^.{31,}$|[^a-z0-9.])

Windows Live Restrictions (hotmail.com, live.com):

  1. 4 to 32 characters
  2. Only letters, numbers, periods, hyphens and underscores
  3. Cannot have the character sequence “fuck”
  4. Starts with alphanumeric
  5. Cannot end with a period (also required by RFC 2822)

The minimum length was just a guess on my part.  It’s not published, but 4 is the smallest free user name I could find.  I’m sure there are words besides “fuck” that aren’t allowed, but you get the point

(?:^.{,3}$|^.{31,}$|[^a-z0-9._-]|fuck|^[^a-z0-9])

MobileMe Restrictions (me.com, mac.com):

  1. 3 to 20 characters
  2. Only letters, numbers, periods, hyphens and underscores

Note: The regex on this page are intended for case-insensitive, un-anchored searches.

Written by Lee Verberne

2008-08-29 at 21:42

Posted in Internet

Tagged with