Tag: Logs

Machine Learning Security and Event Sourcing for Databases

In times past, an application would make an update to a database, essentially overwriting the old data with new data. There wasn’t an actual time element to the update. The data would simply change. This approach to database management worked fine as long as the database was on a local system or even a network owned by an organization. However, as technology has progressed to use functionality like machine learning to perform analysis and microservices to make applications more scalable and reliable, the need for some method of reconstructing events has become more important.

To guarantee atomicity, consistency, isolation, and durability (ACID) in database transactions, products that rely on SQL Server use a transaction log to ensure data integrity. In the event of an error or outage, it’s possible to use the transaction log to rebuild pending database operations or roll them back as needed. It’s possible to recreate the data in the database, but the final result is still a static state. Transaction logs are a good start, but not all database management systems (DBMS) support them. In addition, transaction logs focus solely on the data and its management.

In a machine learning security environment, of the type described in Machine Learning Security Principles, this isn’t enough to perform analysis of sufficient depth to locate hacker activity patterns in many cases. The transaction logs would need to be somehow combined with other logs, such as those that track RESTful interaction with the associated application. The complexity of combining the various data sources would prove daunting to most professionals because of the need to perform data translations between logs. In addition, the process would prove time consuming enough that the result of any analysis wouldn’t be available in a timely manner (in time to stop the hacker).

Event sourcing, of the type that many professionals now advocate for microservice architectures, offers a better solution that it less prone to problems when it comes to security. In this case, instead of just tracking the data changes, the logs would reflect application state. By following the progression of past events, it’s possible to derive the current application state and its associated data. As mentioned in my book, hackers tend to follow patterns in application interaction and usage that fall outside the usual user patterns because the hacker is looking for a way into the application in order to carry out various tasks that likely have nothing to do with ordinary usage.

A critical difference between event sourcing and other transaction logging solutions is the event sourcing relies on its own journal, rather than using the DBMS transaction log, making it possible to provide additional security for this data and reducing the potential for hacker changes to cover up nefarious acts. There are most definitely tradeoffs between techniques such as Change Data Capture (CDC) and event sourcing that need to be considered, but from a security perspective, event sourcing is superior. As with anything, there are pros and cons to using event sourcing, the most important of which from a security perspective is that event sourcing is both harder to implement and more complex. Many developers cite the need to maintain two transaction logs as a major reason to avoid event sourcing. These issues mean that it’s important to test the solution fully before delivering it as a production system.

If you’re looking to create a machine learning-based security monitoring solution for your application that doesn’t require combining data from multiple sources to obtain a good security picture, then event sourcing is a good solution to your problem. It allows you to obtain a complete picture of the entire domain’s activities that helps you locate and understand hacker activity. Most importantly, because the data resides in a single dedicated log that’s easy to secure, the actual analysis process is less complex and you can produce a result in a timely manner. The tradeoff is that you’ll spend more time putting such a system together. Let me know your thoughts about event sourcing as part of security solution at [email protected].

Exercising Care in the Woods

It’s fall and the woods are quite beautiful. For the most part, the bugs have started packing it in, even though we haven’t had a frost yet. I can spend hours in the woods, enjoying a soft breeze, with nary a bite to show for it. There are times where I just sit on one of my stumps up there and wait for something to happen (it usually does). I never run out of interesting things to see in the woods, despite the fact that they really aren’t all that large.

Of course, it’s also the time of the year when I’m cutting wood for winter. So, I often go up with my chainsaw in hand, looking for wood to cut up. The first priority is to keep the woods clean, so I start by cutting anything that is already lying around. Even small wood burns, so I’m not too particular about what size the logs are. Sometimes I find a log that is quite dry and burns nicely lying right there on the ground. In fact, that’s where I found these piece that I cut up.

CarefulWoodCutting01

There is an equal mix of slippery elm and black locust in this case. Both woods burn quite nicely. These pieces are quite dry, but not rotted. Even if there were some rot, I’d take the wood because it’s better to keep the woods cleaned up whenever possible and wood with a little rot still burns just fine.

After I get done looking for fallen wood, I find any snags (trunks that lack limbs) that no one is using. It’s important not to cut down every snag because owls and other birds often nest in them. In addition, it could be a matter of self-preservation because bees will also nest in the snags at times. (For this reason, I actually put my ear up to the trunk and listen for a while.) On this particular day, I found a wonderful piece of black locust to cut up.

CarefulWoodCutting02

This snag looks like a mess. It doesn’t appear to be usable. The inside has rotted out and there are shards where the tree was hit by lightning. However, this is black locust and the wood is actually quite good. Cutting into it, I found that the outside had indeed rotted a little (up to a half inch), but the inside was both sound and dry. so, the snag ended up on the wood pile along with everything else.

On this particular day, I found everything I needed on the ground or as a snag. However, there are some days when I do need to cut a tree down. When this happens, I look for trees that are already completely dry (the bark has come off of its own accord) and no one is using. Even with these restrictions, I usually find all I need. All it takes is a little looking and given the beauty of these woods, looking is something I like to do.

Notice that these pictures show that the woods is intact. It’s what I try to achieve when I cut wood for winter. I leave all of the young trees and anything that’s alive intact as much as possible. Even the ground vegetation is left intact except for the narrow path I create for myself. (All the wood is carried down by hand to minimize damage.) Using management techniques like these ensures that the woods will continue to look beautiful and produce wood well into the future.

Have you taken a stroll through a woods lately? Let me know your thoughts about careful management techniques at [email protected].

 

The Many Appearances of Firewood

Most people are used to viewing firewood as simply cut up logs. The logs are then stacked in cords (128 cu. ft) and proudly displayed outside the home. They have the old homestead or wild west view of wood, with the healthy young male whacking away with an axe and turning perfectly good chunks into kindling. However, real firewood comes in a variety of forms, not just logs. In addition, the wood is often stored in a basement or other area inside the home for easy access and to keep it dry. We actually have firewood in three forms:

 

  • Logs: The old time view of wood cut across the natural growth of the tree. However, unlike the television view of logs, our logs are generally 24″ long and up to 10″ in diameter. No one really takes a huge log and splits it down into kindling (unless absolutely necessary)—they use the copious branches of the tree for that purpose.
  • Disks: A wood stove doesn’t care how the wood is oriented. If you put a piece of wood into the stove, it will eventually burn (assuming the wood is dry). In Splitting the Dreaded Elm I discuss how to avoid splitting elm by cutting the tree into disks that will fit into the wood stove sideways. This means we can burn a tree up to 24″ in diameter without splitting it. Most trees that someone looking for firewood encounters aren’t that size.
  • Slabs: This kind of firewood is actually the focus of this post. Slab wood is what remains when you turn nicer hardwood logs into boards for furniture or other uses. The slabs are bundled together and you cut them to length with a commercial table saw or a chainsaw.


Slab wood is the sort of firewood that you won’t find at your local store and you generally can’t get it delivered by someone who sells firewood. You actually need to know someone who has a sawmill and is willing to sell you the remnants as firewood. What you receive doesn’t really look much like firewood at all. It doesn’t look like a board either—it looks like wood scrap, which is what it is.

Firewood01

The advantages of slab wood are many:

 

  • It costs a lot less than a cord of logs (usually about half).
  • It’s guaranteed dry.
  • The presence of flat surfaces makes it easy to stack.
  • You know you’re getting quality hardwood that won’t clog your chimney.
  • It’s unlikely that the wood will contain any serious pests such as carpenter ants.


Slab wood also has some serious deficiencies:

 

  • It isn’t readily available from most sources.
  • You normally can’t buy just one cord.
  • There is the problem of cutting the slabs to length.
  • It’s absolutely essential that the wood not get wet because it soaks up water like a sponge.
  • You must mix slab wood with other kinds of wood because it tends to burn both hot and quickly.

Our wood pile currently contains all three kinds of wood we use. The pile in the basement of our home has mostly logs and slab wood. The outside pile contains logs and disks. Most of the outside wood currently contains pests, such as carpenter ants. After a good freeze, the ants will be gone and we’ll be able to bring the wood inside a little at a time to burn. In the meantime, we have a wonderful assortment of wood inside to use during the cool autumn months.

Firewood02

So, how do you like your wood (slabs, disks, or logs)? What kinds of wood do you prefer to burn? Let me know at [email protected].