Month: September 2018

Home / Month: September 2018

From a Developer’s Perspective

“NodeJS and Windows don’t work well.”
“I Need to run with Root Permissions to globally install vue on my MacBook Pro!”
“NodeJS broke my Linux Servers FS Permissions.”
“NodeJS can’t be found in my PATH!”

I’m sure you could list ten more items before finishing the next paragraph, but it’s clear to see that when discussing NodeJS, you cannot couple such powerful feature sets without the risk of also introducing issues with your own system or as I like to call it, ‘file-clog’ from the thousands of globally installed modules you make available to each project.

I found myself frustrated with this constant battle, be-it on ANY system that I was using. Eventually, they all became too cluttered and unlike a USB key which you could pull away and forget about, it was hard to clear out the jank without exposing your rm -rf habits to critical file systems. This is where I came up with the convoluted but totally awesome idea: Can I run NodeJS projects through Docker, and discard the container when I am done?

Turns out the answer is yes!

Aside from the above, why would anyone really look into this approach? Well, let me provide a few examples:

  • Your AngularCLI install is out of date, and any attempts to update also messes with your TypeScript version installed in the project or on your system.
  • Your testing framework creates files which aren’t cleaned up after, which results in artifacts on your system which are out of date by the next run.
  • You don’t want to muddy up your PATH with dozens of modules or leave it as stock as possible.
  • You want to leverage similar workflows on multiple computers using a single setup script/configuration.

The two workflows differ due to end-goal, I’ve included NodeJS’ fantastic workflow for ‘dockerizing‘ your application for production and orchestration, alongside my own development workflow. Whereas NodeJS’ simply need minor refinement (such as using multistage Docker builds to reduce final container size -stay tuned for my exploration and updates to that in the tutorial repo outlined below!), my workflow is still a work in progress.

My end goal of reducing .node_modules found on my computer is still not 100%, but it removes the need for global CLI’s and tooling to be existent on my file system, alongside Node itself. I imagine at this point into the post, you’re wondering why would I bother trying to complicate or remove the NodeJS dependencies in my workflow; to which I simply say: why not? In my mind, even if the workflow gets deprecated or shelved entirely, I’m glad that I got the chance to try it out and evaluate the value it could provide.

Dockerfile – My Workflow Tutorial – Development

My workflow leverages Linux symlinking with your application folder purely for the ability to edit your project in the IDE or text editor of choice, instead of in the container. This, coupled with many CLI’s having auto-build enabled by default creates a powerful and automated development engine. Scripted portions allow for simplistic automation of the redundancies such as mounting the code directory to /app, and all that is left is for you to run in the container (which the script lands you in):

Leveraging the Seneca Application in the Offical-Node-JS-Way Folder

One critical item which you need to do is enable in the Docker settings the shared volume location for your work/development directory. Without this, the script will fail to copy a symlink version of your project. On Windows 10, this is still an uphill battle where permissions and file system securities make this a not-as-smooth process. See the following link for an explanation why the bash script determines your OS and changes location prefix:

Running run.sh puts us directly into the container with our code in /app

The beauty of this method, in my opinion, is that it allows for consistent environments (such as what Docker is intended for) both for development and testing, with the absolute minimum of clean up or exposure to your filesystem. Because of the system link to your project folder, files modified in your editor or in the container reflect the other. This leads to node_modules also being a residual artifact that my next version of this workflow aims to remove from the equation. Once you’ve shut down the container -and thus, removed the link to your project(s), a container-cleanup is as simple as:

Or to kill all running containers

Then to finally remove the image

Or to remove the image(s)

Development Server working via Port 8080, utilizing a Node module such as Nodemon would cause rebuild and refresh (client-side) per code change. Damn useful!

And boom, you are now back to a clean filesystem with your project safely and cleanly in its own place, the only remainder being the .node_modules folder in the project itself which you can delete manually.

Dockerfile – NodeJS Tutorial – Production Build

The official way that NodeJS recommends using Docker is for containerizing your final application, which I highly recommend once you get to said stable state. It’s fantastic for having all your final dependencies and compiled source code running in a container that can be handed off to the cloud for orchestration and management.

Running build.sh to pull Docker image

I used this method quite often when deploying MEAN-based microservices at SOTI, and also for my own projects which are then orchestrated with Docker Swarm, or Kubernetes.

Configuration and listing of Docker images

The benefits with this workflow include being able to utilize Docker’s multi-stage build process so that the node_modules prior to being bundled in the application exists in a staging container which is never included in your final image, and instead, only the final output is bundled.

Local test of final application container

Taking their tutorial, I wrote two scripts titled build.sh and run.sh (pictured above), which automate some more of the process. Taking an old, lightweight application written for OSD600 and leveraging Express as a dependency, you can see how powerful this option for bundling and containerization is!

Closing Thoughts on V1

Well, I hope you get the chance to utilize this workflow and improve upon it. I’m looking forward to seeing what you can do with this little experiment of mine, and also how it may better maintain the health of your host operating system while exploring the hundreds of JavaScript frameworks and libraries which are created daily!

I decided to label this Version1, implying that when I have the chance to revisit the process I’ll update and improve it. In that vein, I also did some thinking and decided to compare or share some thoughts on both processes:

  • Following the NodeJS way is far too costly since it would recreate the container each time, the current workflow at least keeps all global cli’s to the container itself, and Node itself is contained as well to the image.
  • Likewise, following the NodeJS direction would remove some of the modularity I was aiming to keep, so that it could be used on one, three, ten projects all the same.
  • I had Toto’s Africa on repeat for a good hour or so while drafting this, apologies if you can notice any rhythmic mimicry to the first verse at points in the writing.
  • Updates to this will come, but for now I am content with the current workflow despite shortcomings and complexity.
  • Docker’s documentation is by far one of the best pieces of technical writing I’ve ever been exposed to. It’s that good.

Tell me, what is your Docker workflow? How do you use Docker outside of the norm?

Tutorial Repository

References & Research:

https://nodejs.org/en/docs/guides/nodejs-docker-webapp/
https://github.com/raygervais/OSD6002017

Troubleshooting:

http://support.divio.com/local-development/docker/how-to-use-a-directory-outside-cusers-with-docker-toolbox-on-windowsdocker-for-windows

City Bokeh
“City Blur”

So, this blog post has been long overdue. There is both so many experiences and thoughts I want to share, and yet so few which I personally feel would be of any use to you. Regardless, without any order, here are some of the activities that I’ve enjoyed and also learned from this summer. For the technical, programming centric, let me follow that up with a smaller post since I didn’t commit anything major this summer outside of my work at ManuLife (which has its own lessons including Docker, Kubernetes, Concourse, Chef, … let’s write an article on that soon, okay?).

PS. TLDR can be found at the bottom.

Photography

It’s no surprise to many who’ve followed my social media channels outside of Twitter that I’ve started adopting many vlogger / cinematography based habits including in-the-moment streaming, hourly updates, and capturing images in any way I can while trying to express a dramatic composition. Through social media, I also complained about small annoyances (such as not being able to change the focal length or aperture on my Pixel 2 XL. First world problems?) which probably should have been simple annoyances, but instead led me to purchasing a second-hand Sony Alpha A6000. Out of all the impulse purchases made, I think this is arguably one of the most spontaneous, and also best purchases I’ve made without hesitation. The camera enables both a new level of expressive outlets, and also a new money-swallowing hobby for me to jump headfirst into.

Svitlana likes to use the camera as well
Svitlana likes to use the camera as well

Yet, I didn’t once hesitate or draw regret over the purchase or the hobby. Instead, I began planning more and more events, trips, areas to visit with frequent friends such as Svitlana, Jessica, and Nick. I began spending countless hours learning about framing, editing RAW images, and also how to manage a basic following -all through the magic of YouTube, Udemy or Pluralsight (Thanks ML for hooking me up with PS for free!). The visits to the hometown became much more meaningful, and allowed me to stretch the concepts I was learning where possible. I spent two hours waiting for a chipmunk (named Fred fyi) to come and go, getting closer and closer to the camera until finally I captured shots such as this.

Fred the Chipmunk
Fred the Chipmunk

My interests in Photography also helped both my father’s media and branding, and my friends who’s social media accounts were itching for new profile pictures, content, and edits. I suppose the creative side broke through quite a bit over the summer thanks to photography. Here are some shots which are either my favorite, my friend’s favorites, or ones being withheld from Instagram for now as I figure out the direction and tone I’d like to push towards -assuming there is energy, passion and time in the day to account such. Leaving it as a hobby is equally a likely scenario, which I’m content with as well, because it enables a new way to capture memories such as upcoming trips and events.

Me!
“Staged”, Taken by Nick

Nick
Nick Guyadeen

SvitlanaSvitlana Galianova

Videography

I’ve always been a huge fan of the Tech YouTuber / Content Creator niche, and have felt a genuine connection to them similar to how one may with Twitch Streamers or Bookhauls (two other content-based digital niches). Likewise, I always had a small voice in my head who enjoyed cinematography of different genres, different moods; good or bad. I sometimes found myself coming up with frames, transition ideas which at the time, I thought were a sign that I was thinking of UX improvements on whichever project I happened to be working on.

Toronto Island Sunset
Toronto Island Sunset

Todd Folks - Thunder in the Hills
Todd Folks – Thunder in the Hills

As YouTuber’s became more prevalent in my life, I began to shift some of my ‘idol space’ over from musicians to these new faces. Marques Brownlee, Jonathan Morrison, Michael Fisher, Karl Conrad, Kai Wong, Peter McKinnon; the list could go on. Regardless, with my foray into photography I figured this is the perfect stepping stone into videography as well. Who’s parent doesn’t want their child to be a vlogger? (sarcasm). In all seriousness though, I remember during my first software development COOP in Haliburtion the opportunity to edit some videos for a client who required audio work, reframing etc to be done. I volunenteered since I was the must comfortable with linear timeline editing and audio production, and to this day I still have the Google Keep note that I made which simply said ‘I really love video editing’. It was a thought which at the time seemed so perverse, I had to jot it down and see what I thought down the road. Guess all things come back in time if they are meant to?

Fireworks #1
Fireworks #1

Fireworks #2
Fireworks #2

Building a PC

Major shoutout to my roommate Jack, who helped choosing the parts which enabled the following comforts as I threw my wallet directly into a morgue with a DNR taped to the front:

  • 4 year minimum future proof
  • 4K Video Editing capable
  • 10+ VSTi3’ / VST3’s per channel in a +32 channel project (320 VSTs) <- Looking at you Ableton Live Set
  • Upgradable DDR4 Memory to 64GB
  • Power two 1440p screens or so down the line

Specifications:

PCPartPicker part list

CPU: Intel – Core i7-8700K 3.7GHz 6-Core Processor

CPU Cooler: Cooler Master – Hyper 212 EVO 82.9 CFM Sleeve Bearing CPU Cooler

Motherboard: MSI – Z370 GAMING PLUS ATX LGA1151 Motherboard

Memory: G.Skill – Ripjaws V Series 16GB (2 x 8GB) DDR4-3000 Memory

Storage: Samsung – 850 EVO 500GB M.2-2280 Solid State Drive

Storage: Western Digital – Caviar Blue 1TB 3.5″ 7200RPM Internal Hard Drive

Video Card: Gigabyte – GeForce GTX 1070 8GB G1 Gaming Video Card

Case: Phanteks – Enthoo Pro M Tempered Glass (Black) ATX Mid Tower Case

Power Supply: EVGA – SuperNOVA G2 750W 80+ Gold Certified Fully-Modular ATX Power Supply

Sound Card: Asus – Xonar DGX 24-bit 96 KHz Sound Card

Wireless Network Adapter: Gigabyte – GC-WB867D-I PCI-Express x1 802.11a/b/g/n/ac Wi-Fi Adapter

Songwriting / Recording

This one should come to no surprise to any of my peers, since I’ve had this off and on again relationship with music production, songwriting, and the idea of expressing one’s self through noise.  With the building of a production-grade computer (workstation I’ll continue to use term wise, since I do hope to employ many technologies / containers for orchestration), I’ve also rediscovered a drive to make music. This drive I felt died years ago, where even my SoundCloud (subtle plug of older material!) displayed a stagnated cut off between such passion, and eventually what my idea of being an ‘adult’ was. In other words, priorities changed, music was thrown out to make space for studies and relationships. I did manage to record a small cover that I talked about earlier in the year with some friends while living in Mississauga, but even that experience felt like a forced effort at times. That can be found in a separate account which will become my primary I imagine:.

2007 Fender Stratocaster
2007 Fender Stratocaster

Music is limitless genre wise, and there are many items that I’ve dabbled in the past and also on my Guitar / Piano recently. I’ve grown in interests and also listening preferences, often jumping even further into the spectrum of previous interests:

  • The usage of ambience for both foreground and background textures.
  • The removal of instruments to provide more power to the few playing.
  • Layering different parts instead of layers of repeated motifs.
  • Not striving for the ‘Analog’ sound where it doesn’t need to be.
  • Allowing songs to be simple tangible four chords structures, instead of 17 chord theory-based monstrosities which I dubbed Progressive House appropriate.
  • Allowing a song to be described as ‘lush’, ‘dark’, ‘moody’. This helps to drive the tonality instead of strip away from fear of being too ‘emotional’ lyric / sound wise.

I could probably go on, and I’m sure I’m also forfeiting many better, outstanding notes I’ve made in the past. You get the idea! I’m excited once I find a good balance post-summer, to create and share through here and more conventional mediums and networks. I already have a few instrumentals and lyric-driven songs waiting to be worked on that encapsulate some of this interesting year.

Manulife Tower
Manulife Tower

Summing the Above: Content Creator

While discussing offline with some friends, they’ve managed to piece together quite a bit of what my final focus of the above will look like; which is the interests above can and will be joined at times into a single project: a music video, short film, or for vlogging / content-creation related mediums!

Fire Up North
Fire Up North

Nick's Mixtape
Nick’s mixtape cover

Bonus!

I decided during this long weekend to give the above idea a try, creating a rough song template / mood, followed by (all B-Roll) footage which could paint a small mood. Utilizing the time that I wouldn’t be annoying my roommate Jack with loud music or constant swearing when shots ended up out of focus, here is what I came up with as an one-day-release experiment! All was captured, created and released in a single day (including the music!).

Waiting – Demo Concept

TLDR: Disregarded common hobbies / passions in exchange for social experiments, creative outlets, allowed hobby programming to take backseat.

PSS: I feel rusty having no written a blog post in a while, so as things hopefully improve and soar to better heights, I hope you don’t contract a sickness while reading this one. It’s another off-the-rails style, with very little preplanning / scripting and instead following one’s train of thought.