Drupal 8 Migration

Posted on

I have resisted migrating this site to Drupal 8 for years. Every time I looked into it there was some killer feature that D8 couldn’t do. The times I attempted testing out the process it was always a complete disaster. Well, time is running out on Drupal 7. The old system is slated for end-of-life in 2020 and it’s now or never for the “upgrade.”

In this post I am going to recap my experience in migrating a fairly simple Drupal 7 site to Drupal 8. Hopefully, someone facing the same issues as me will come across this post and will find something helpful.

Long story short, the migration process sucks bad. If you don’t have a little experience with directly manipulating a database or troubleshooting software in general, don’t even attempt it.

I am not a command-line guy. This was going to be a drush-free migration.The whole push in Web development towards command-line apps for everything has been one of the worst things to happen to the Internet. It is virtually impossible for a beginner to just jump in and start creating content. Developers have made huge strides in interface usability and yet the tools to create those interfaces are nightmares of horrible UI design puzzle-boxes that exist mostly as job security for entrenched Web devs. Thank Shatner for the few out there like Prepros.io who are at least trying to make new technologies user friendly.

The first hurdle for me was finding a decent local development environment that could handle Drupal 8. My old standby, Xampp, has become garbage in recent versions with no 64-bit support and no interface improvements. I found Wampserver64 and it has turned out to be fantastic. Most importantly, it allows me to run two different local domains, which is required as the migration process works by sucking an old site into a new one rather than overwriting an exiting Drupal 7 install.

After setting up a clean Drupal 8 install with all the migration modules installed I set about creating a plan of attack. Oh, as an aside, whatever you do, do not install the Migration Example modules. It will fill your new site with a bunch of garbage content and fields that don’t get removed when you uninstall the module. My first order of business was to determine which I my D7 modules had D8 versions. A spreadsheet and lots of notes proved helpful. I tried to limit only the most integral modules and skip any ones that are just cosmetic. A lot of popular modules have been added into the core, but in many case they are watered down versions (Menu Block stands out as particularly bad).

Migration Begins

At that point I clicked all the buttons, watched and waited. It looked like most of the content came over but there are big problems with text-filters. If they don’t match with the new D8 filters, the content won’t render. That is really dumb. I had to log in too phpMyAdmin to manually change values. The next big problem was that my post were peppered with strange characters. This had to do with my Sql server not having correct character encoding. This article explains the fix you should do before migrating (I had to search and replace for hours because I failed to do this). Basically, add the following to mysql.ini:

[client]
default-character-set = utf8mb4

[mysql]
default-character-set = utf8mb4

[mysqld]
character_set_server=utf8mb4
collation_server=utf8mb4_unicode_ci

Comments

The next big problem were comments. Only the most recent comments were rendering and I couldn’t see any difference between those and the bad ones. There appeared to be two tables for comments in the database: comment_field_data and comment__comment_body. Turns out if the langcode column doesn’t match between these two tables, the comments don’t render. More stupidity. Had to manually add “en” to the fields. Thankfully this site doesn’t have that many comments.

Page Caching and Dev Environment

The next step was setting up the local.settings.php to disable caching and render theming helper code. A couple YouTube videos explained this pretty well, although I still am having to flush the cache to see changes half the time.

Views, Oh God Views

Views is a fantastic module. It’s the reason we all use Drupal in the first place. So, why the holy hell aren’t views migrated by default?! So freaking annoying! I had to recreate about 15 custom views manually in order to get many of the site features back. Don’t delete that Drupal 7 site yet. You will definitely need it for reference.

Turning on the Modules

I then started installing and enabling various modules. Like I said earlier, things are different here in Drupal 8. Menu_Block is missing features, Menu_Position is just plain broken (a fix is in the works), Redirect is flaky and throws errors.

Everything to do with CKEditor is horrible. TinyMCE was orders of magnitude better. More robust, more advanced features, better looking, better integration with Drupal… sigh. I have a feeling I am going to be using code view a lot with CKEditor.

Pages of Fun Theme Redux

If there is one area where Drupal 8 is leaps and bounds better than D7 it’s the new Twig-based theming system. The syntax is far less cryptic. A simple {{ content.field_myfield }} renders a field. Perfect. My theme is far simpler than before, yet looks pretty good. Lots of fancy CSS layout tricks here that probably fail in Internet Explorer.

Launching the Site

So, after about two weeks of poking at it, I finally launched the site on December 19th. Within about 30 minutes I got my first spam comment. I do miss Mollum. I’ve tracked down a few anti-spam modules, but it’s going to be tough. I gave up on comments on the Nonagon.us site long ago and this may be a losing battle. Out of my cold dead hands Russian bot farms!

And here we are, a couple days in and things seem to me working okay. I’m sure I will need to upload fixes very soon. There are always sections of the site I miss. And when my first round of analytic come in I may need to start hunting down broken links. In the meantime, click around and enjoy. (And buy my art!)

Pages of Fun: Now with Drupal 8

Posted on

Well, I have finally made the jump to the newest version of Drupal. We are now running Drupal 8 and, man, was it a pain in the butt to migrate the old site. I will forever recommend WordPress over Drupal to anyone who doesn’t want to pull their hair out every time a new patch is released.

Still, there is a lot to like about the new system. I will soon post the gory details of my migration experience, but, in the meantime, let’s hope this works and that I am not swamped with comment spam.

GNOG on PC (7/10)

Posted on

GNOG is a cute little puzzle game in which you are left to click and poke around without any instructions and without knowing what your goal is. This is a little frustrating at first, but once you realize that you are trying to open up the big cartoon heads, things start to make sense. Or as much sense as opening up floating heads makes.

There are some common tactics: right-click to flip the box, figure out how to pop the back open, then things get weird after that. The main draw here is the surreal art and soothing music. It’s not that hard and it really just feels like a point and click adventure with no story. The game does get some negative marks for crashing on the end puzzle (with no way to get there without starting over) and for not working if you have Citrix receiver installed on your PC. Talk about weird.

One Cut of the Dead (9/10)

Posted on

Excellent Japanese zombie movie that isn’t quite what it seems. Avoid spoilers and just watch this. The title refers to the fact that the movie is built around one continuous take of a movie crew filming a zombie movie. Of course, eventually there are real zombies and we know the rest, right? Maybe not. The level of detail here is astounding.

Ruiner on PC (7/10)

Posted on

Ruiner rages with a riot of red. So much red. It’s like an entire game based on a Photoshop filter applied to the R channel. I went in to this thinking it was going to be your standard twin-stick shooter like Nex Mechina and I was excited. No, this is one of those aim with the mouse, steer with wasd games. That control scheme always seems a little bit off and, when you’re in an isometric view, the aim never quite matches with your perspective. On top of this there are several special moves bound to other buttons. The most important of which is a chainable dash move. This is very awkward when you need to use you mouse both for aiming and positioning your dashes.

About two levels in to the game I was getting flashbacks to Until I Have You, which has similar cyberpunk aesthetics, but also has a game ruining control scheme. I gave that game a bad review on Steam describing it as being like controlling a game with a theremin. The developer then contacted me wanting me to elaborate. Awkward. That’s just about as weird as when Sweetwater Music calls you in regards to your online purchase of strings and a few guitar picks. There’s customer service and then there’s just leave me alone so I can never leave my desk and buy things on the Internet and rant anonymously in the middle of the night.

By that third level Ruiner had become so frustratingly hard that I was considering giving up. Then, in a moment of inspiration, I realized, “Hey, there’s an easy mode!” Upon switching down the challenge the game actually became fun. The whole point of the story is that your some cyber-modified human with unstoppable kill-powers. When you die every few minutes, you lose a bit of that key plot point. With the tempo is slowed down, I was able to wrap my reflexes around the janky controls and actually start to feel like I wasn’t just clicking randomly. Maybe the game could have used a more gradual difficulty curve to make normal mode actually playable to an old geezer like me. I recommend it. It’s short, über stylish, and spazzy fun.

Mister America (7/10)

Posted on

If you haven’t followed “On Cinema” on Adult Swim, I can’t imagine appreciating this movie. If you’re a fan, you will love this. It’s ninety minutes of uncomfortable anti-humor in which Tim Heidecker presents himself as one of the most unlikable characters in cinema history. Cinema buffs like Gregg Turkington will immediately recognize that the mockumentary thing has been done many times before and much more effectively. There was a ton of potential here for satirizing the electoral process but it is mostly just a character study of an awful human being who never actually saw Sully. Would have been vastly improved by more On Cinema On Location expert historical analysis.

Assassin’s Creed III on PC (4/10)

Posted on

This game has been sitting in my to-do list for quite some time. I got it as a freebie from Uplay, and to be quite frank, that may have been too expensive for this clunker. I suppose if you really love the parkour mechanics of the other games you’ll be entertained by this, but I am long-since over jumping from towers into bales of hay. With a handful of naval missions, some of the seeds of the vastly superior AC IV are here, but, to mix my metaphors, the mechanics still a bit half-baked (wait you can bake seeds, so maybe that metaphor works… or is it a simile).

The story is so dull and pointless. It feels very constrained and linear. There a many side quests and missions but they aren’t very fun. I skipped all the gambling, hunting, crafting and homestead building; all of which felt like such a boring waste of time. I never upgraded my character and was able to finish the game without any problems. And I hated all the modern day sequences. Again, pointless.

Transferring Patches to a Yamaha DX100 with Sysex and a PC

Posted on

On this site, I had previously posted a set of 24 Yamaha DX100 synth patches for download. The DX is a notoriously difficult synthesizer to program. That difficulty extends to its load/save workflow as well. Originally, the DX100 came with a special cord with a MIDI-style plug on one end and three headphone-sized jacks on the other. This is meant to be plugged into a data cassette recorder. The red cable is audio out, the white is audio in, and the black (I assume) controls the tape player. You hit a button combo on the DX100 and then a screeching modem sound is output to the tape. Modern users can record this sound on a computer as a .WAV file and it will work the same as and old-school tape deck. This works okay, but there is a better way to archive and reload sounds.

Meet Sysex

Sysex stands for system exclusive and it’s a part of the MIDI standard that allows synth manufacturers to define their own specific message formats. In our case it is useful because the DX100 uses sysex to store and retrieve patches. With some free software and a cheap hardware interface, you can easily store and retrieve synth sounds on your PC as both individual patches and a complete 24 patch bank.


Requirements

A Yamaha DX100 Synthesizer

A DX27 should also use the same data, but I have not tested that.

A MIDI Interface

I opted to go the cheapest possible route and bought a Roland UM-One from Sweetwater. This cable is a USB connector on one end and MIDI In/Out on the other.

Connect your DX to the PC and all the sudden you can control VSTs with the DX or send sequences from your favorite DAW to the synth. I have Windows 10, and this worked perfectly with no special set up or drivers needed.

Bome SendSX

This is a tiny, powerful software utility that does two things. It reads and sends sysex data. You can actually see the data flowing in as you play the keyboard. Your playing is translated into beautiful hex values instantaneously. Download Bome SendSx for free!


Saving All Your Patches at Once to PC

On the Yamaha DX100 user patches are saved in the Internal bank of 24 patches. These are typically accessed by pressing the internal button and then any of the 24 green numbered buttons. To save all 24 of these patches at once do the following:

Step 1: Connect The Hardware

Connect your DX100 to the PC using the MIDI interface

Step 2: Enable Sysex

On your DX100 press the brown Function/Compare button.DX100 FunctionYou’ll know you are in function mode because there will be an uppercase F in the left side of the screen. When function mode is enabled, the number buttons correlate the function listed in below each button.

Press the Sys Info key (green key #5).

DX100 SysInfo

The screen should say Sys.Info:on. If it is set to off, press the +1 Yes key to turn it on.

DX100 Yes

I think it’s pretty safe to just always leave this on.

Step 3: Get Ready to Transmit MIDI

Press the Sys Info key again until the screen displays Midi Transmit?

Step 4: Set up SendSX to Receive Data

Back on your PC, open SendSX and, under Options, make sure that MIDI Thru is unchecked.

Click the Clear In button to erase any data in the IN window.

The Midi In pane should be empty and SendSX is ready to receive data.

Step 5: Transmit Your Patches

While the screen reads Midi Transmit? on your DX100, press the +1 Yes button to transmit your data to the PC.

DX100 Yes

And don’t touch anything else on your DX! You will now see a bunch of hex values in the Midi In pane in SendSX and a success message.

Step 6: Save the Data on Your PC

From the SendSX File menu chose Save Midi In to save your patches on your computer as a .SYX file (or TXT if you want).


Saving a Single Patch to PC

Saving a single patch is a little easier. The simple act of switching patches instantly transmits that patch’s sysex info over MIDI. Just make sure to follow steps 1–2 above to turn on Sys.Info:on. Detailed steps are as follows:

Step 1: Locate Your Patch

On your DX100 find the patch you want to save. It can be any of the 192 factory patches or any of 24 user-programmed internal slots. Let’s say I want to save internal patch 14. I press internal then button 14.

Step 2: Set up SendSX to Receive Data

Same as Step 4 above. Open SendSX your your PC and click the Clear In button and insure MIDI Thru is disabled

Step 3: Transmit Your Patch’s Data

Press just the number key of your chosen patch again. In my example, I’d press button 14 again. As before, don’t touch anything else on your DX after hitting the number button!

You should now see a bunch of hex values in the Midi In pane in SendSX and a success message. The hex string should be shorter than before (around 101 bytes).

Step 4: Save Your .SYX File

From the SendSX FILE menu chose Save Midi In to save your patch on your computer as a .SYX file as you did with the complete bank save.


Transferring a Set of Patches from your PC to the DX100

Transferring patches from your PC to your DX100 is also handled by SendSX.

Step 1: Prepare SendSX for the Transfer

Open SendSX and click the Clear Out button to erase the contents of the Midi Out pane.

Go to File then Open to find and open your saved .SYX file.

Opening the file will populate the Midi Out pane with hex values.

Step 2: Turn Off Memory Protection on the DX100

On your DX we need to turn off Memory Protection mode by pressing the brown Function buttonDX100 Function  and then the Memory Protect button (number 12) .

DX100 Memory Protect

The screen will probably read M.Protect:on. Switch this to off by pressing the -1 No Key.

It should now read M.Protect:off. From this point on be careful because, with memory protection off, it is possible to delete or alter saved patches.

Step 3: Transmit Your Patches

In SendSX click the Send(F4) button. It doesn’t matter what you are doing on the synthesizer. If memory protection is off, the Internal patch bank will be replaced with the saved sysex data.

Step 4: Turn Memory Protection Back On

Turn Memory Protection back on to prevent an accidental erasing of patches. Press Function then Memory Protect then +1 Yes to reenable it.


And Finally… Loading a Single Patch to the DX100

Follow the steps 1–2 above to load your single patch into SendSX and turn off Memory Protection on the DX.

Step 3: Select Your Patch Location

Get your DX100 ready to receive data by selecting the internal patch location that you want to overwrite. Press Internal button then any of the 24 number buttons.

Step 4: Transmit the Data From SendSX

In SendSX click the Send(F4) button. The DX will now have the patch loaded and selected in the slot location from the previous step. If you play the keyboard you will hear your loaded patch, but…

Step 5: Save Your New Patch on the DX100

Your patch will not be saved just yet. If you switch to another patch, the sysex loaded patch will vanish from memory! You need to save the patch in the same manner you would save any edited sound.

Press Function button then the reddish Store button and select the internal slot in which you want the patch saved. This will probably be the same number that you selected in step #4.

For example the screen will say Mem Store 2→?. Press the Number 2 button and the patch will be saved in internal slot #2.

Note: You don’t necessarily need to be in Function mode to save a sound. You just want to be sure you are not in Edit mode because then the Store button becomes the EG Copy button.

Step 6: Turn Memory Protection Back On

Turn Memory Protection back on to prevent an accidental erasing of patches. Press Function then Memory Protect then +1 Yes to reenable it.

In Conclusion…

Well, that was a lot of info. Trust me it’s not that difficult. I just like to break tutorials up into very detailed steps to make sure you understand everything you are doing. Now if you want to download some of my custom DX100 patches download the .ZIP below which contains all the sounds from this post as both a complete 24 patch bank and 24 individual patches.