Hooking Cachegoose to Redis on NodeJS

On my last post I outlined the steps required to enable caching using the Cachegoose NodeJS module. I thought it would be a fun exercise to add a Redis caching to the node-billz application. Previously I had implemented an in-memory cache which works fine for just me, but in an actual production environment memory could become an issue.

Cachegoose makes it very easy to add the Redis storage to the application since it uses the NodeJS cacheman plugin under the hood. Slight modifications to my main app.js file were required to pass the host, port and password to Redis. I also added the USE_REDIS flag to my .env file so Redis support can easily be toggled on or off without any code changes. Redis integration complete.

Another plus with using Redis is being able to explicitly set the cache eviction policy. The cacheman-memory module follows LRU (least recently used) eviction policy. Redis allows multiple eviction policiesso it allows some flexibility.

Once deployed to the production environment I needed to be able to verify the cache was working correctly. I love using the cli but some things are just easier to visualize, uh, visually. Doing some quick searches I found a few open-source projects out there that were pretty nice, but Redsmin really stood out from the rest. I am using their free plan which allows monitoring of one Redis instance which is all I need.

Going forward I may revert back to the in-memory cache but it was a fun exercise adding Redis support with a few lines of code.


Caching with NodeJS/Express

Working with payees and payments in node-billz I basically end up loading and reloading the same data on every page. The list of people paid each month will rarely change. The list of payments will grow over time but generally only a few payments are made per week. Basically, these are perfect candidates for caching so I thought I would see how difficult it would be to get this implemented in Node.

Searching GitHub I quickly found a few cache providers that would work with MongoDB and more specifically Mongoose since that is what I am using to work with backend data. There were a few false starts with downloading orphaned projects that were no longer supported, but finally I settled with cachegoose. It has been updated recently plus it gave me the option of an in-memory cache as well as caching with Redis and others.

The cache provider module selected, the next step was to get the app configured to use the cache. Two lines of code later everything was working. It does not get much easier. I opted to use the in-memory cache for the time being, but you can easily point cachegoose to a Redis server at initialization.

The final step was to wire the cache to the queries. You can specify the number of seconds you would like to cache a query or you can cache it indefinitely with a custom cache key. If a cache needs to be purged, call the clearCache method with the custom cache key to clear it. I opted to go the second route since the data only changes occasionally. When a payee is modified or a payment is made I clear the cache keys so the data can be re-cached the next time it is queried.

It was incredibly easy to setup caching, aside from the false steps I had at first. Given the size of the ecosystem though I suppose I should not be surprised to find some dead projects out there.

I would be interested in learning about any other libraries or tools you guys are using. Feel free to comment or drop me a line.

Welcome to 2018

2017 was a very quiet year for me as far as blogging has been concerned and that is something I intend to remedy in 2018. I thought I would go ahead and kick things off since there is no time like the present.

While things have been quiet I was actually quite busy in 2017. The first part of the year was spent picking up Angular(JS/2/4/5). That kept me occupied for the early part of 2017. Next I wanted to look in to ReactJS because I have heard so many good things about the framework. Turns out it is not the easiest framework to pickup so I decided to put it down for a bit.

After working with frontend frameworks for the first half of the year I thought it would be great to move javascript to the backend and decided to pick up NodeJS and ExpressJS. I will not really expand on this too much but overall I have really enjoyed my time tinkering around (have you checked out node-billz, my NodeJS-based bill calendar)? Be on the lookout for more posts coming soon.

As far as non-tech things, I did not read quite as many books in 2017 as I had intended. I got started on the Wheel of Time series by Robert Jordan and I made it to book seven, I hope to finish the series this year. Also Raymond E. Feist, arguably my favorite author of all time, has a new series coming out in March.

2018 is going to be a great year!

Writing a bill calendar with Node and Express

Everyone has their “go to” application when it comes to learning a new software stack. I believe every programmer out there has written the venerable “Hello World” application at least once in their lifetime. Todo apps were all the rage when Angular came on the scene. Some folks I know like writing blog applications. My app of choice is a bill calendar.

It may seem like a strange starting point when it comes to building a learning application, however date manipulation is one of those things that seems very simple on the surface but it is deceptive. You get that down and you can tackle almost anything.

The first time I wrote the bill calendar was with ColdFusion(Lucee) and ColdBox, but I thought it would be really cool to write a single page application using AngularJS. Alas, Angular is great on the client side but if you would like data to persist it requires a back end server. A bill calendar that requires you to enter everyone you pay every time the page is loaded would get monotonous quickly. I could easily write an API in ColdFusion to handle this, but it did give me an excuse to play around with NodeJS.

So here we are

The very first thing I did was to find a starting point. I could have very easily started with a bare-bones implementation of Express and built on top of that, but the NodeJS community is awesome so I did not have to reinvent the wheel. I found the hackathon-starter boiler plate app and was able to quickly get up and running. It almost feels like cheating since Express and much of the middleware has already been done for you, but at the same time it follows best practices and gives you user authentication with social sign-in from the gate. It is possible to jump right in and starting coding with some of the community driven boiler plate apps. If this one was not your cup of tea, simply pick another one!

Now that the foundation has been laid it is time to start to actually build something. The hackathon-starter app breaks the layout into several logical folders: config for your Passport configuration to handle user authentication, the controllers folder that hold the actual logic for your application routes, the models folder that contain the mongoose schema for items and finally a views folder that will hold our PugJS templates. App.js in the root of the folder structure that is the main file for the application and pulls everything together.

The next step is figuring out the structure of the data to retain. Since this is a bill calendar the name, a short description, due date, amount and the website URL would be fantastic to track . There are several other fields related to payees that are retained, but those are the important ones.

I also created a schema for payments so a payee can be marked as paid in the forecast view. If the payment amount changes each month we can also show the end user what the average payment amount is for that payee. That will require us to track the id of the payee, the id of the owner, the date the payment is due as well as the amount.

Once the data has been setup the real fun begins. The forecast is what really ties all these pieces together. Forecast.js in the controllers folder only has one route handler, the getIndex() function. This method will calculates the start date, gets a list of bills for the current week and will then proceed to the next three weeks.

Here is the overall flow of how the whole process works:

  • Load the user’s current list of payees and build a schedule for the current month. Also get a sum of the total amount due each month.
  • Load a user’s payments to see if it corresponds to a current bill and mark it paid
  • Using MomentJS, determine the first day of the current week based on the user’s settings. By default the first day of the week is Sunday, but allow the user to define that value if they would prefer a different start day.
  • Now it is just a matter of using MomentJS to determine if the due date falls in between the start date and end date. If so, add it to the weekly list and add the amount to the weekly total. Also look in the payment information to see if I can match a payment to the bill due date so it can be marked paid.
  • This process continues for every week displayed on the forecast view.

Wrapping it all up

I am not the strongest JavaScript developer, but the time I invested playing around with AngularJS really helped. Being able to write ES6 without having to worry about transpiler configuration to compile it back to ES5 is fantastic. Everything just works.

Overall my first real experience with Node/Express was very positive. The community around NodeJS in general is fantastic and if I hit a road block I could easily find an answer to my questions. I am looking forward to polishing up my bill calendar app, it has been a pleasure so far.

(Originally posted on Medium)


Creating a background process in Lucee

Lately at work I have been using ColdFusion event gateways to accomplish a few very specific tasks. I thought it might be cool to see if I could leverage CFThread on Lucee do accomplish some of the same things.

The first step was writing a working demo with ColdFusion. A buddy was kind enough to point out that using an event gateway with a cfthread running inside of it was doing extra work for nothing. Event gateways were introduced before cfthread so I went ahead and just refactored it back as a regular ColdBox service. Everything was running smoothly in ColdFusion.

Next was trying to port the same code to Lucee since that is what I tend to run for personal projects. The one road block I did run up against is cfthread terminating at 50 seconds, the default request timeout for Lucee. It turns out there is a simple fix for it, but it made me scratch my head for a while since I didn’t run into it on ACF. The code with fix is below.

It was not a hard fix (setting the request timeout extremely large), but it did take a bit of google-fu to figure it out. Interestingly enough, setting the request timeout to zero did not work.

Goodbye 2015

It seems that 2015 is coming to a close and I thought it would be a good time to compose my fourth and final post of 2015. It has been a slow year blogging but that is not to say I have not been busy.

I have been working almost exclusively with ColdFusion almost 20 years now which is almost half of my life as scary as that is to say. This year has been focused on learning new methods and technologies. I spent quite a bit of time this year picking up the NodeJS stack. As a front end developer I have written a lot of jQuery interacting with the DOM. Using javascript to write a server just seemed plain awesome. Moving forward I have a feeling many of my personal projects will be built on top of NodeJS.

After my crash course on NodeJS the natural progression of things led me to AngularJS. I will have to say this one was a real pleasure. After so many years of using jQuery to modify the DOM after it was rendered it was refreshing to use javascript to actually render the DOM. I have been splitting my time between working with Angular 1.x branch and also trying to pick up Angular 2 since it is getting closer to a stable release.

Finally I have been working with GitHub’s Electron to build cross-platform desktop apps. Mostly this has been writing tools I can use for work since that would actually push me to learn Electron and Angular. Writing an application once and being able to build it for Mac/Windows/Linux is just awesome. My teammates at work have enjoyed the fruits of my labor.

Aside from that I have played a lot of games in 2015. I enjoyed Destiny for a while, but really my old standby is Diablo 3. I am excited to see what 2016 has to bring and I hope all of you have a happy and safe new year!


My Node/Express/Angular adventures

Lately I have focused a lot of my attention on Node.js, Express and Angular. One of the great things about Node is how extremely flexible the language is. However, when you are learning a new technology that flexibility can be a liability.

My first attempt at learning Node.js and Express was focused implementing framework paradigms that I would use developing a ColdFusion application. My application was a Diablo 3 character differ that would allow you to compare two characters side by side. It was a good lesson in how to structure a pure Express application. Plus, it worked, which was awesome.

Next I decided to focus on Angular and since I was trying to run it on a MEAN stack of course the first two projects I checked out were MEAN.IO and Mean.js. Both of these are really great projects, but when you are just trying to pick the stack up I was just completely lost.

When that did not pan out, I tried getting started with Node and Express using the same framework I used with the character differ. I was using Handlebars for server side render engine on Express which unfortunately conflicts with variables in Angular. Yes, I know you can change the brackets with $interpolateProvider, but I did not want to muck about changing things when I shouldn’t have to.

I was a little annoyed, but I was just going to develop the Angular portion of the app served with Nginx and write the API portion in Node/Express. I was taking a look at John Papa’s Angular style guide though and many of the configuration blocks just falling into place. Following John’s style guide along with his ng-demos repo I had my new Node/Express/Angular app up and running in no time.

If you have not checked those out, I highly recommend it. I might head back to MEAN.IO or Mean.js eventually, but I am enjoying learning the new stack for now.