Switching servers *now*

This post might not actually successfully post.  Right now…  I have set the server up to be moved…  But…  It might happen before this has been posted.

Posted in Uncategorized | 2 Responses

Initial steps moving server to linode

Last night I started to work on getting this site moved over to linode, and have some comments to make regarding:

  • pricing structure
  • responsiveness of admin site
  • responsiveness of updating server images
  • server locations
  • image availability
  • process with regards to gentoo on linode

So let’s go and look at those topics at least at some level.

Pricing Structure

With our current host (GANDI) we love the granularity of the pricing structure.  If I have something which is very processor heavy, but doesn’t use much hard drive space, or memory, I can easily build a machine with that focus in mind.  That really is a great way to get systems which are really ideal for specific purposes.

The problem there is in order to provide that kind of granularity of pricing, you end up with a situation where it can become difficult to actually handle all the complexities of setting that up.  On GANDI, we are currently spending about $125-200/month on the hosting for 3 different sites (on 3 different servers).

So, while I initially saw the linode pricing structure of machines based on set bundles as being a bit of a problem, looking a little closer I think that even for the same server sizes, there is about a 20% reduction in price, and I think that their current smallest servers will do well enough (right now) for current needs for each server.

Responsiveness of the admin site

Repeatedly when working on the GANDI site, I have wondered if my click had gone through, and I simply was waiting for a response from the site, or if I failed to get the click to it.  With linode, I have had no such issue.  The server is so responsive that I actually wonder sometimes if I did actually do anything because rather than my usually expected “clear change” something minor just pops up, and is responding to what I asked it to do.

Responsiveness of updating server images

Here is something which I felt was really bothersome with GANDI.  When I updated images, I ended up frequently feeling like I might as well go off and do something entirely different for twenty or more minutes.  I knew I couldn’t really do anything with the machine for that long.

On linode, I am finding that even though I manually shut down the server, manually make the changes, and manually reboot the server, I can usually do so with a minute or less for the entire process.  It also is very clear about where in the process that is.  So I would say even that will make it at least feel snappier rather than “operations in process” which doesn’t really tell me anything.

Server Locations

One of the issues with GANDI which has lead to me feel I really have wanted to move away from them, is that the servers which I was running in Baltimore are getting shut down at the end of November, and that was where I had them located.

I might have been not overly bothered by this and less inclined to move the servers if they provided an option for a more or less “one click” migration, or better yet given the heads up, allowed people to migrate as they wished (one click migration if they wanted to just move the existing image etcetera, but manual migration if they wanted to modify the server), but their migration is entirely manual, and one issue I repeatedly ran into was when creating a new server, or new disk or whatever, it defaulted (for me at least) to the Baltimore data centre, and that setting was not really noticeable enough that I made sure I was doing the right thing.

Linode has servers in North America, Europe, and Asia.  Further, on creating all of my server instances, I have pretty much noticed where they are.  Once my instance was in the wrong US city, but not a big deal, in part because when I notice it, I can recreate it in another datacentre in about 2 minutes (probably less).

While I’d really like to have my servers in Canada, I can’t say I’ve been able to find any host which isn’t really expensive to handle what I am wanting with this in Canada.  Which doesn’t mean it doesn’t exist.

Image Availability

The server images which are available on linode are pretty diverse.  There are 9 of the top Linux distributions including my favourite gentoo which really surprised me.  I saw that the gentoo image was on the old side, but I went with it, and that may have been a mistake, but after having the server up and running for more than a day, mostly working on getting it to “current gentoo” (most of it the issue that gentoo always takes a long time to setup) and having already figured out about 3 different issues where the install is not working correctly, and why, I have decided to continue to plow ahead.

I do not recommend gentoo unless a person really wants to get into the depths of Linux, or if they have some issue with the politics of certain big names along how certain things are done (systemd comes to mind) as gentoo tends to be for power users.  While it does end up compiling pretty much everything that can be (certain packages are split into a -bin and the usual with no -bin as they are such large packages) and this is claimed to potentially improve performance, I am not entirely convinced that is indeed the case beyond minor improvements for most users.  Though if you are using certain older hardware or whatever, gentoo can allow you to do things you may not be able to on other systems (not tried it on any of the iMacs we have as they have all either been “in use” or “dying”).

Gentoo Server Creation

The gentoo server image which is available on linode is almost 2 years old from what I can tell.  Gentoo does not really do “releases” in the sense that a given release is using a given repository, there is a common repository for every release.

This works great when gentoo is maintained relatively frequently, and you usually end up with hardly any issues most of the time.  When working with gentoo on a system which hasn’t been maintained, you will often have to manually handle a number of issues, in order to get everything up and running in a reasonable manner.

In doing the updates on this, I have run into a number of issues, each one required me to look at why things were not updating the way that I wanted.  First off the first couple of updates were not even attempting to do anything as one package ended up blocking another package.  This can be a difficult situation to figure out, if any of the packages are in the “packages being updated list” (command line options, which can include either explicate packages or one of two (or maybe more) “sets” of package lists) it can be tricky if you try to uninstall a package listed there, especially when using the @world or @system keywords as if the package is listed in either, bad things can happen which are hard (very hard) to back out of once you’ve gone and changed certain things.

Then once I managed to get the system so that it is at least attempting to update @world, I quickly run into problems that certain packages can’t be updated as they are blocked because the package management system doesn’t support them (the system is called portage) so in order to update those packages I’m going to have to see about updating that first (no problem).

Now I am getting a list of all the packages that need to be updated on the system.  This isn’t looking too bad at all.  So I go ahead.  I notice some aren’t updating, but decide that for now I can just skip them.  It actually gets most of the system updated.

Now let’s try again.  Sometimes the issue is as simple as the fact that packages got updated in the wrong order, and just restarting after ending at the “nothing to do” will just fix it.

Sadly, no…  So I see what I need to update (and the failing package is a key package (glibc)).  Oh nice clean error message saying that the version of gcc (also a key package on gentoo) doesn’t reliably compile glibc.  Simple.  Just update that…

Once done, try to update what’s still there…  And right now, it looks like we’re good to go.  At least in terms of having the system up to date.  Loads of configuration I’ll need to work on.

Patreon Supported Post

This post is supported by my Patreon supporters.  If you like this, and would like to see more, I’d love to see you head over there and let me know.  Even “support” me there.  Amanda Palmer may be someone who says things I can’t really say in the same ways I can.  So there is a link there to something I feel she has said in a way that I really can’t.

Posted in Business News, Site News | Tagged , , , , | Leave a comment

Gentoo QEMU/KVM install

I am currently working through the Gentoo QEMU guide and so far have rebuilt my kernel and rebooted…  Now…

I am currently updating my @world with the command:

emerge -uv --newusue @world

And this latest update has 10 packages. Still not emerged quemu and am seeing a couple of use flags I’d like to enable before doing that. sdl2 and gtk.

Now that we have the emerge running (with some graphical front ends), we are waiting for 33 packages to install.

As of now, I will leave this here.  Too tired to go further.

Posted in Uncategorized | Leave a comment

Developing ebuild for Gentoo

I am currently working on the development team for the accounting package LedgerSMB and I have got to a stage with that development that I am looking at getting a working ebuild (or possibly several, due to dependency issues) for LedgerSMB on gentoo.

These are the steps that are probably going to have to be done (though temporarily things can possibly be done somewhat different order):

  1. Create a development image of a clean install of gentoo.
  2. Create overlay for LedgerSMB
  3. Test Test Test…
  4. Make Changes
  5. Repeat from 3…

I will be documenting these in time.  It looks like I may well be looking at the development virtual machine in short order.

Posted in Uncategorized | Leave a comment

Some updates about our activities…

This is a quick post about what has been going on here.

  • Site downtime
  • One site moved
  • Expiring SSL certificate
  • Need to move two other sites
  • Possibly moving another site

Site downtime

A bit of bad news first.  Two of the sites we host (the two which need to be moved still) went down for 4 hours yesterday.  It appears that the hosting company had issues with their data centre which caused a number of sites that they host to go down for up to 5 hours.  There was at least partial downtime for two of our sites (including this) for about 4 hours.  At one point in that 4 hours I checked the sites out, and was able to get some activity from them, but not the usual level that I have been used to.

One site has been moved

One of the sites which I was hosting the same place as these two which went down, got moved.  I will probably write a more complete post about doing that shortly.  It ended up being a bit of a pain for a number of reasons.  It may have actually been easier to have moved to an entirely different hosting provider.  But now that I have done that, and have ironed out many of the issues which caused so many problems with doing it, it should be easier with the next two.

SSL certificate here is expiring shortly

The secure certificate for here is expiring shortly.  My decisions as to how I will handle that will probably end up being a bit of playing around with different ideas.  I am thinking there are probably two ways which I may go.  With a certificate issued by my hosting provider, this is what I have been doing.  Or, I might end up going with a certificate from Let’s Encrypt.

The last time I installed a new certificate, I installed one from Let’s Encrypt.  With doing that, even though it was for a different type of service (Matrix Home Server) I feel I probably could do so again.  I’m not sure if that will be any different from using the hosting provider’s.  It appears that it might actually be easier in many respects.

Need to move two more sites

This site, and the Open Psychology Project site, still need to be moved.  I am hoping that moving this site will allow me to reduce my “credit consumption” by at least 25%.  Though it is possible that the other one will increase consumption a small amount.

Possibility of moving a client site

I have a client who has had a site hosted on WordPress.com and I would like to see if we can move the site soon enough to make it worthwhile.  Client initially said that they would like to move it, but then changed mind.  I’d like to know what they would like to do with the site, but it seems that I haven’t been able to get this conversation started, at least not in a very helpful way.

Posted in Site News | Leave a comment

Accounting Software

I have been in the process of testing an install of the accounting software which I currently am running.  The version I have been running (in part due to the fact that I am not really in need of it that much) has been in beta the whole time.

I have been running LedgerSMB.  This is an open source software which is mostly written in Perl.  I am running the current version on a gentoo box, and it is working well enough for my needs as far as I can tell right now.

My needs have been fairly modest really, mostly I want basic bookkeeping, which as far as I can see it does, the ability to invoice clients, which it does, and the ability to receive payments, whether or not this is working I’m not sure.

I will be looking to see if I can get things working on a new install of the system shortly.  I’m not quite sure how I will end up handling the new system.  I am not really interested in wiping out the existing system which I’ve been tweaking and building for some time, so I will either be looking at doing a dual boot, or finding some way to create a gentoo virtual machine.

Future Plans

We are looking at getting some things with this working better:

  • Install on “clean” system.
  • Create an ebuild which will do that.
  • Ensure existing system really is meeting needs.
  • Transition the ebuild for the 1.5 beta, to 1.5 release (once that happens).
  • Start working on ebuild for future 1.6 beta.

That really looks like a fair amount of work, but this in part depends on a lot of the work which ends up being done on the development team.  I understand that my end of things will allow the team to work better on seeing what is, and what is not working.

Build on clean system

This has a few different things which I will need to handle before managing to do this:

  • Decide on way which to install it:
    • Virtual Machine
    • Dual Boot
  • Create new system which is “clean” to install from
  • Research what needs to be done to create an ebuild.

Decide how to install

I am currently thinking that getting a new Gentoo system up and running in some virtual machine, would be the best option, as a dual boot, would mean I would end up losing access to the existing system, while I am working on the new one.

Virtual Machine

That reason is probably almost enough to be looking at getting it on a virtual machine.  One potential disadvantage is that often on virtual machines certain aspects actually end up being handled by the hypervisor rather than by the computer hardware itself.  Though I don’t think this will be an issue.

Dual Boot

I see a number of issues here.

For one, as already mentioned, it means booting into one or the other system at a time.  Which removes access to the existing system, while the new system is running.

Another issue which I am thinking about, is that this will really need to have a separate partition, disc, or something which to boot from.

Create new “clean” system to work from

Due to the fact that I am likely looking at potentially working on a number of different systems to work from, I will probably create a nice “base” system which I can then clone, and then build new test suites on it.

I am not really sure where this will go from there.  I will likely have to decide on what virtualization environment to work with.

Right now, I feel that this is about where I can start looking at this process for now.  So the future information about this will end up going in other posts.

How we are funding this

Currently the content is funded by a combination of means.  One clients who are paying for the work which we have been doing, have been part of how we have managed to keep things up and running and we love all the clients very much.  Unfortunately due to a lack of client base, and a lack of being able to draw new clients the content creation has been funded by our Patrons on Patreon.  If you become a patron through there (for as little as $1.00/thing, with so far never more than 4 things per month) you will be able to further support our activities.  To reach our next pledge goal (while I’m writing this) I believe we only need an additional $7/thing pledged.  It would be super awesome if you did that.

Oh, and a note on “patreon only content”.  Currently I am noticing that a good, and rather important thing is to actually produce content which people really can access.  That may change as there is more patreon support, or I am producing more content period.

Posted in Business News, Computer Support | Leave a comment

Moving Servers

My hosting provider sent an email which let me know that they are shutting down this datacentre, and that I will have to move the servers to another datacentre.

Currently there are 3 servers which I am running out of the datacentre which they are talking about closing, this one, the “dmp-develop” server, and the “open psychology” server.

My current plan is to go through the process of moving the dmp-develop server first.  I don’t know if doing so is going to end up being the best, as it is the one which is most different from the other two.

But, it is also the one which is least critical if it happens to “go wrong”.

I do not know if doing so will end up being an easy thing, or not.  There are going to be a bunch of things which have to be in place for each server.

But it could be as simple as “moving the disk, and setting up the server on the new datacentre, and then setting up DNS to point to new server.”

Well, it hopefully can be done with some variation of that.

 

Posted in Site News | Leave a comment

Photoshop Photomerge

I took a bunch of photos today, and a couple of the “sets” were done with the intention of stitching them, and getting them to work.

The first one (that I did) was only two images. Here is the video of it.

Here is the final result…

Photoshop stitched of peeling bark

This is the result of the stitching. I need to get the other two in a bit.

These are the two pictures I started with. At least I hope so.

Peeling birch bark

Bottom picture which I ended up stitching.

More peeling birch bark

Top picture which I stitched in Photoshop.

This post was supported by my Patreon Supporters, if you would like to become one, then please just click and “support” me.

Posted in Graphic Design, Photo Editing | Tagged , , , , , | Leave a comment

Setting up SUDO Gentoo

I had not been using SUDO on my testing server until just a little while ago (like less than a few hours).  I realized that this is a problem which I should fix, so I did.  Well mostly…

Installing SUDO

After trying to see what I had to do to get SUDO working correctly, I tried a simple test to see if SUDO was installed in a way that it was working at least to the point of finding it, with the sudo command.  It didn’t.  So I decided that it needed to be installed.  With my unprivelaged user I did:


$ emerge --pretend sudo

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] app-admin/sudo-1.8.15-r1  USE="ldap nls pam sendmail -offensive (-selinux) -skey"

The package wasn’t installed, I checked out the USE flags. Decided not to change any of them.

I then looked and did the install with the root user.


# emerge --ask sudo

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] app-admin/sudo-1.8.15-r1  USE="ldap nls pam sendmail -offensive (-selinux) -skey" 

Would you like to merge these packages? [Yes/No] yes

This went as it usually does. So, that means it took a while to do, but it was successful.

Enabling SUDO For Requested Users

With the command visudo I edited the sudoers file so that the following lines were used:


root ALL=(ALL) ALL
%sudo	ALL=(ALL:ALL) ALL

With this I expected adding the user to the group sudo would allow that user to use sudo. So I found to do that, I would need to add the group sudo and then add the user to the group. As that group didn’t exist already.

# groupadd -r sudo
# usermod -aG sudo username

This should have allowed me to sudo. So I tested it, but it didn’t work. It complained that I wasn’t allowed to use the group from username. So, I changed it to remove the :ALL which context highlighting wasn’t really understanding. Still no luck, so I added the user individually. Now it works. But not how I would like it to.

I will look into what might be a problem with this. For now, we currently have a working setup, but it’s a bit along the “too hard coded” line for my feeling it is actually fully working.

Posted in Computer Security, Computer Support | 1 Response

Securing Server (Part 1 of ?)

There was a recent link posted to YCombinator Hacker News titled “My First 10 Minutes on a Server” (Link to article, link post is here).  With seeing that it reminded me that I have meant to start working on securing the web test server (which is currently not web accessible).

There are some things  mentioned here that I thought were useful.  Those are the following items:

  • Root Password
    • This partially extends to user passwords
    • Also involves issues of logging into root account
  • SSH setup
    • Require that people use keys
    • Testing if logging in with key works
  • SUDO
    • Setup SUDO
    • Test that it is working
  • Firewall
    • There are some things to look at here:
      • IPTables vs. IPChains (I may be way behind on things here)
      • Using shortcut services such as ufw
    • Kernel support
  • Updates
    • Having some degree of automatic updates is very important in keeping things secure
    • Gentoo uses the Portage package management system, finding how best to handle this could be difficult
  • Blocking “probable attack IP addresses”
    • It is recommended (and nothing wrong I can see with this) to use “fail2ban” to automatically create rules.
  • Use of Two Factor Authentication
    • I recommend this for any service storing critical information (which surprisingly is pretty much every one).
  • Logwatch
    • Have something which monitors your logs

All of these things really will need to be dealt with in detail I think in future posts.  I will try to deal with them more clearly in the rest of this post to give a bit of an idea where I am going in the next little bit with securing the server.

Passwords

I am not entirely sure what I think of the mess of suggestions about passwords out there.  There generally are passwords which no one can argue are hackable.  These passwords are next to impossible to remember, are very long, and are made of a mixture of letters (upper and lower case), numbers, and special characters.

These are exactly the kinds of passwords you would like to use for a high security situation such as a root password on a server which has a web facing interface, especially one which would allow root access with that password.

The problem with such passwords, it is pretty much impossible for someone to actually get one which really meets these requirements in a manner that they will be able to be remembered.  If you consider a password which is 15 or more characters long, which is composed of random characters, most people will not be able to remember them.

That requires a password manager of some sort to remember them.

Using Password Managers

I have not looked very far on the different password managers that people use.  I am not all that happy with the solution I currently have.  There have been issues with security of it, and with any of the “big name” password managers, they are a high-profile target for being attacked.

Part of the reason that there are big efforts on the point of crackers to attack particularly password managers is, that if they do manage to crack a particular user account on a password management site, they often will gain access to dozens if not hundreds of passwords from a single user account, accessing the whole database, will open things up a great deal more.

With LastPass they say that the passwords which are stored on their site, are stored in a manner which would require your password and username to recover them.  So, provided you have an uncrackable password as your master password, you should be safe.  But what does uncrackable mean?

Uncrackable Passwords

The idea that some passwords are uncrackable is based on the idea of password strength.  The majority of “password strength checkers” are deficient in the sense that the password, “password” is as “strong” according them as “dpraosws” which of course you are probably thinking that is a little silly to consider them both the same.

The reason they treat them the same way, is because the majority of them only look at the number of characters of a given type.  So a password “strength” metre will look at a given word as the same strength as those same letters which are random (or near random).  While, I believe that they are a good “estimate”.

The problem with this simplistic approach is that it treats each letter individually.  If you are using anything approaching a “dictionary word” you are using something which would run on a “first pass” of cracking.  Cracking passwords almost never just throws random passwords as tries.

Those lists of “most common passwords” are based entirely on that idea.  For the most part, even with access to the entire database, you can’t actually pull the passwords out of it, (you shouldn’t be able to) but only test a given stored “hash” against a password.  Many sites actually thrown an additional layer on top of that, in that the password isn’t what is hashed, but there is at least one additional layer, which requires access to that bit of information as well to even test passwords.

So, what really are uncrackable?  Long strings of random (looking) characters with a “broad character set”.  A password like “!a5#C356C2345asfcedfr@#” is pretty much uncrackable (I wouldn’t suggest using any static string such as that particular one as there is probably a good chance that some crawler will add that particular “word” into crackers’ dictionaries) given that no one can realistically “find” it without too much difficulty.

Password managers are almost all able to create strong passwords.  Some might have a simple “strength” setting.  Others may have multiple variables which are used to generate the password.

I intend to look deeper into the question on password managers in the future, and what ways you can generate passwords which are secure, but sufficiently memorable to be able to use for a master password for your password manager.

Setting Up SSH

SSH (Secure SHell) is a way to log into a machine and get an interactive shell session.  There are a number of ways which you can set up how you can log in.  The most common way to log in is with a password.  There are a number of issues with this, which include that you have to remember the password, and that if you can easily remember the password, chances are low that you will be using a very secure password.

Another way to log in, is to use a “key pair”.  This is generally considered more secure.  And once it is setup, it only requires that the person just connect.  Unless there is another layer as well in place.

I agree that at least for remote connections, you probably want to set things up to reject password login attempts.

For now I am going to leave that discussion at that.

Setting Up and Using SUDO

SUDO is a way to execute a “single command” as another user.  It has become more or less a replacement for SU which allowed you to “become” another user.  The reason for using SUDO over SU is that it means that you only will be executing commands which require higher (or different) privileges as that different user.

If you have created a directory with a bunch of temporary files in it, and are trying to delete it, when you type a command which tries to delete every file on the system, at worst the command will delete all files you have rights to delete.  Which could well be way more files than you intended, but usually it would not break anything major on your system.

With SUDO setup, you can run a command by prefixing it with the command “sudo”.  It will then require a password before doing anything more.

The whole setup for SUDO is probably a whole blog post itself.  So I will leave things here for now.

Firewall

As I mentioned above, there are at least a few different ways to handle firewalling on Linux (I am running Gentoo as the distribution of Linux I am using).

There are three different systems which are available (as far as I can see) which are IPTables, IPChains, and IPFWADM.  My understanding is that IPTables is the newest version of it, and IPFWADM is the oldest.  They are all part of the IP Masquerading system.  IPTables was introduced around 1998, so it is relatively mature.  As of 2002 there was some talk of it not being mature enough for some use cases.  I do not know if there is any reason to believe this is still the case.

With this in mind, I will be looking at setting up IPTables I think for this server.

There are different ways to help automate this, which I haven’t looked into. The article I heard mentioned was ufw, and it looks like fail2ban.  Will look into both.

Setting up Automatic Updates

I am not really sure how this will be done.  The place which I saw this information was talking about Ubuntu/Debian, and this will be dramatically different as Gentoo uses a different package management.  I have set a couple of jobs up to handle this.  One which should give a list of packages which can be updated, and another which updates the “portage tree”.

This isn’t automatic updating, but it is getting closer.

Two Factor Authentication

I don’t know how this will actually work.  This is something I will have to look into.  Two factor authentication is adding an additional level of requiring a device to get a code to enter before being authenticated.  This may be something which will happen fairly quickly.

Logwatch

Logwatch is a way to monitor logs.  There probably is more to do with this than simply installing it.  Right now, that will be the last of this post.  Changes will be made to link to the other pages with this.

Posted in Computer Security, Computer Support, Network Security, Site News | Tagged , , , , , , , , , , | Leave a comment