Book Review: Hadoop 2 Quick-Start Guide: Learn the Essentials of Big Data Computing in the Apache Hadoop 2 Ecosystem

Having never installed or played around with a Hadoop environment myself, I was on the look out for an intro style book that would give me the basics and enough info to start me off.

When browsing this one caught my eye as I didn’t even realise there was a Hadoop 2 and the title was pretty much spot on for what I was looking for so decided to give it a shot.

Overall, I enjoyed the book and it was spot on for what I was looking for. It’s a traditional tutorial/walk through type of book on how to get a Hadoop cluster up and running and how to admin/interact with it, but it also covers enough theory that you don’t need to have any prior experience with Hadoop to follow along.

However, I would say that I think it’s overpriced in the paper edition and retail price ebook so if you’re interested in this book, try and read it on Safari or get a Kindle edition to make it affordable. Other than that definitely recommended.

The book starts off with a really good overview of what Hadoop is, the MapReduce pattern and the changes in Hadoop 2. Good intro material.

The next chapter is a more traditional walk through on how to install Hadoop uses both the Hortonworks distribution and the Apache sources. It also covers use of Ambari for a simple web based admin console for your cluster. Nothing too detailed is explained here as it’s covered off later, but it’s a straight forward walk through so is spot on for that.

The third chapter gives a really good intro to how HDFS works, covering the nodes involved, their roles and the approach taken to replication and then some basic file system commands. I particularly enjoyed this chapter as I hadn’t used HDFS before and so some of the concepts around the different nodes, compute following data, append only files and block sizes were spot on for what I needed to understand.

The forth chapter covers running jobs and monitoring them in the web gui, along with some examples for base lining the performance of the cluster.

The fifth and sixth chapters walks through the MapReduce approach to data analysis, using word counting in text files as the main example and then moves on to the basics of writing code to create MapReduce jobs, covering the basics in Java and Python. Simple and straightforward, but again spot on in term of depth.

The seventh chapter runs through some of the other Apache tools within the Hadoop ecosystem, covering Pig, Hive, Sqoop, Flume, Oozie and HBase. These are just quick overviews but interesting as I wasn’t aware of some of these.

The eight chapter is really nice in that it focuses exclusively on YARN (Yet Another Resource Negotiator), which is new to Hadoop 2 and is one of the big differences in the new version. It walks through how to use YARN for things other than the traditional MapReduce pattern, using the YARN distributed shell as an example, before touching briefly on how some of the other Apache tools can be used with YARN.

The last two chapters focus on admining Hadoop through the commands required and the Ambari interface. I skimmed these as I’m only doing a very basic setup to get my head around Hadoop but would look back to these as needed.

In summary, the author notes initially that this book is written to a "hello world" level in terms of depth and that’s spot on across the book. It gives you enough info to get you to a working example, and then it’s up to you. I really liked this analogy and it’s exactly the level I was looking for. I also liked the author’s style of writing so will also be going looking for more of his book to find some more advanced material on Hadoop.

If you looking for an intro to Hadoop that’s a nice combination of both theory and high level tech implementation, then this is definitely worth a read.

One thing I would say is that I got through the book very quickly (3 hours roughly), and was surprised to see when I checked Amazon that the paper version is just over 300 pages as it really didn’t feel like that. It reads more like a book of around 150 pages, which in my head makes sense for quick start book.

Why I highlight this is that while I really enjoyed the book, as I mention earlier, I don’t think it’s worth the price of $27 that the paper version is currently retailing for. For me it’s more in the $15 - $18 bracket and so if you’re going to read this then definitely try and go for the Kindle edition which is worth it at $17.

Links:
Amazon: http://www.amazon.com/Hadoop-Quick-Start-Guide-Essentials-Addison-Wesley/dp/0134049942
Safari: https://www.safaribooksonline.com/library/view/hadoop-2-quick-start/9780134050119/

Book Review: Creating A Data-Driven Organization

I stumbled across this book while browsing and it’s title obviously jumped out to me as I'm always  interested in anything to help quantify analysis or build data driven approaches to what I do.


I wasn't entirely sure what to expect but in summary, it's a really enjoyable, easy read on how to build data-driven teams and the culture to support them in an organisation.

The book starts off by establishing what the author really means by data-driven, touching on some of the fundamentals of data quality, collection and analysis.

After these initial chapters the book really got interesting for me as it starts to look at the organisational and cultural consideration of building a data-driven program.

The author first outlines the different skillets required for a rounded data-driven analysis team, covering skillets like business skills, programming, devops, stats, visualisation, machine learning and big data analysis. I really liked how the author shows these as complementary skills across the team, but highlights that your team don't need to be experts at all.

One really nice aspect is that the need for strong visualisation is hishlighted immediately, specifically in relation to it’s role in not just performing data analysis, but selling it the rest of the organisation. This is further later on in the book through a whole chapter on visualation, including how it can/should be used effectively, covering a lot of the ideas from Tufte, etc in a really nicely summarised form.

The author then moves on to describe the different types of data-analysis, how they are used and then works through some discussion around metrics and A/B testing as core examples of how data analysis can be applied to business contexts.

The next three chapters cover what I think to be the most important aspect of the whole book; the approach of decision making and it’s effect on data-driven approaches, the key comments of a data-driven culture within an organisation and the role of the C-suite in establishing this culture. These chapters outline many of the key cultural challenges to moving towards a more data-driven approach and are great reads for anyone who may be pushing for more data analysis within their organisation, but it struggling to get traction.

The book finishes out with a chapter on privacy, ethics and risk, which obviously as a security guy I love to see. I particularly like the “ick” factor approach that the author outlines to dealing with data analysis and privacy.

Overall I think this book is a great introduction to a lot of topics relating to data analysis and data driven decision making, and incorporates some really good lessons on organisational structure, culture, skillets and challenges with adopting data-driven approaches within organisations.

The author highlights thoughout that this book doesn’t touch on the tools or technology used for data analysis, or details on data analysis approaches, as these are covered in many other books, which are referenced at needed. So if you're looking for this type of material, definitely go elsewhere.

However, if you’re new to applying data-driven approaches to your field (IT, business or otherwise) or if you’re a manager or leader looking to understand how you can affect change within your organisation towards a data driven approach, I'd highly recommend this.


Links:
Amazon: http://www.amazon.com/Creating-Data-Driven-Organization-Carl-Anderson/dp/1491916915
Safari: https://www.safaribooksonline.com/library/view/creating-a-data-driven/9781491916902/

Book Review: Building Microservices

Continuing my up-skilling on cloud security, I wanted to get a better handle on application architectures that map into cloud computing patterns and while micro services aren’t a cloud specific architecture, the key goals of loose coupling, high scalability, etc align well to a cloud environment so I figured this would be a good book to have a a read through.

The first two chapters are very easy reads, covering an introduction to micro services and their benefits, mapping strategic goals to principles and practices.

The next chapter introduces the fictional MusicCorp organisation and application that is used throughout the remainder of the book, demonstrating the concept of bounded contexts and how to apply it to a monolithic application. At this point the book really gets into more detailed discussions on the topics, with each of the further few chapters being pretty meaty in comparison to the earlier chapters. The rest of the chapter covers some of the key technologies that can be used to facilitate micro services (RPC, SOAP, REST, XML, JSON, message queues, etc) touching on both the positives and negatives of each and also covers area like versioning, choreography/orchestration and integration with COTS.

The author then expands on the MusicCorp example and uses it to demonstrate how to split out the application into multiple micro services, before moving on to CI topics like deployment and testing and a further short chapter on monitoring. For me the chapter on breaking a monolithic application into micro services wasn’t as relevant for what I was looking for, but it some of the high level approaches were interesting to understand how it may be tackled.

Security is touched on next but as the author mentions early on in the book, he’s not a security specialist so this is a fairly light chapter covering authn/authz and SSO, touching on OpenID Connect and SAML One nice thing to see in this chapter was a call to be frugal with data storage in light of potential data loss events, particularly where personally identifiable information may be in play. Nice touch!

The last main two  chapters cover system/organisational design and micro services at scale, both of which I thought were great introduction on the topics.  Too many organisations think concepts such as devops or micro services can simply be tacked on to their exist structure, but this chapter does a nice job of dispelling this myth. Chirstian Posta wrote a really good blog post on this specifically related to micro services and I'd also recommend Mike Cohn's chapter on team structure from his Succeeding with Agile book.

One aspect of the book that I really liked was the liberal use of links to other material and books when further more detailed explanations are merited. This avoids the author going off on tangents, which I often find many authors doing (sometimes as a necessity to explain a concept… and sometimes just to pad the book).

I’m not a developer or application architect so at times the book goes slightly into too much detail for what I needed, but to be fair this only rarely happens and so doesn’t detract from the overall flow. Of course that probably means for someone who is a developer or application architect that it won’t go into enough detail, which sometime that is held out by other reviewers.

From a security perspective, I’d highly recommend this book as a great way to get up to speed on how applications should/will be deployed in the cloud and microservices in general. Additionally, if you're still working in a very specialised/siloed organisation, this should be up there to read to understand how things may change. Ultimately, if your organisation isn’t doing something in this space now, then they will soon and you may as well be up to speed!

Links:
Amazon: http://www.amazon.com/Building-Microservices-Sam-Newman/dp/1491950358
Safari: https://www.safaribooksonline.com/library/view/building-microservices/9781491950340/

Book Review: Python And AWS Cookbook

I’ve been playing around with AWS properly for the last couple of months and had mainly been getting myself up to speed with the key security considerations, the console, the various services AWS provides, while playing around with setting up an ELK stack in AWS.

I’ve also been interested in playing around more with Python so figured this would be a good opportunity to combine both and get a better understanding of AWS and Python at the same time.

This book introduces you to the Boto Python interface to AWS and walks you through a series of very simple examples of how to use it.

The book is primarily split into two section; one covering EC2 and the other S3. Both sections cover all the basics that you could look for, including how to enumerate the EC2 instances/S3 buckets in your account, how to loop through regions (if needed), how to create new instances or buckets and how to edit tag, metadata and such. The book also covers some basics of ELBs, security groups and S3 permissions so basically, most of what you’d need to do some basic scripting of EC2 and S3.

The book briefly touches on CloudWatch and SNS but nothing too in-depth, nor does it cover any of the other AWS services Boto current supports (See here for the current list).

A lot of people have criticized this book as only touching the surface of AWS, and that's definitely true.. you can know nothing about Boto at the start and get through this book in around eight hours (probably much less if you know Python beforehand.. which I didn’t). However, at the end you’ll know enough to get and running and can then loop back to either the Boto or AWS documentation to fill in any more gaps.

If you’ve just used the AWS console and haven’t tried you hand at the API, then this is a perfect intro to the nuances that exist with the APIs and ultimately you’ll learn way more about AWS because you’ll start seeing options or constraints in the API calls that you may not even realize exist (or at least I know I did!).

While there’s nothing much here that you can’t get directly from the Boto documentation, I always like following a book along as opposed to jumping around read-me docs so if you’re similar, and looking for a book to kickstart your understanding of Boto and to help you put together some basic scripts for AWS, I really recommend this.

I'd love to see an updated edition of this book, as it was released in 2011 and things around moving so quickly on AWS, it would benefit from a refresh and also some more examples added.

Links:
Amazon http://www.amazon.com/Python-AWS-Cookbook-Mitch-Garnaat/dp/144930544X/
Safari https://www.safaribooksonline.com/library/view/python-and-aws/9781449308100/

edX Economics of Cybersecurity Course Review

I've been keeping an eye on the area of economics and information security for around eight years so when I saw this course pop up back in November I signed up immediately for January, despite not really knowing what to expect.

If you work in information security and are a fan of the Freakonomics series of books/podcasts, then the ideas used to analyze info sec in this course will be right up your street so just go and sign up now for the next session!


If you're not sure what on earth economics of information/cyber security is, then have a quick read of this paper and it'll give you a much better intro than I could ever give.

The course itself is the usual style of MOOC with recorded video sessions along with discussion forums, live webinars and some multiple choice questions at the end of each section.

The course material is split over six sections covering the following topics, with each section around an hour in length and with an accompanying webinar of a further hour:

• An introduction to economics in the context of security;
• Measurement of security; 
• Security investment and management: 
• Market failures and 
• Human factors in security.

In terms of content, I thought that the material was a fantastic introduction to a wide range of aspects of economics of security and pretty much spot on for the level of detail I was expecting. I would have loved to see more detail but have to appreciate that it's an introductory course!

I found that based on my existing reading in the area, I was very familiar with the majority of the content in the course, in particularly the areas around the fundamentals of applying economics in security, measuring security, investment and risk management and behavioral heuristics/biases. However, the section on policy interventions and privacy definitely gave me some new insights.

In terms of pre-requisitions for the course, I feel that if you had never done micro economics or had any exposure to the area of economics before, then it'll probably be a bit of a shock to the system on the first week as they very much dive straight in! Because I'd read a lot on the topic, albeit in a completely unstructed way, I was pretty familiar with almost all of the topics covered and with some basic background in economics I was able to keep pace no problems.

I also felt that some of the sections could have had more context set initially to lead people from a traditional, technical information security background in. For example, the human factor section jumps straight into explaining the reasons behind poor decision making by individuals, but doesn't really explain where in information security you'd normally see these kinds of poor decisions being made. For more experienced info sec professionals, they'll immediately understand the context in relation to either risk management decisions or end user opinions, however for more junior people, outlining the examples up front in simple terms would greatly benefit the course.

I was a bit disappointed with the multiple choice questions at the end because when you got answers wrong, there was no way to get prompts as to what the right answers were and you only get the description of what the right answer was when you get the answer right... So in the end I found myself attempting to brute force the answers for a number of questions, just to understand why I go the question wrong!

I really enjoyed it personally as a refresher in the area and also learned some new aspects that I hadn't come across in the areas of market failures and policy intervention and privacy. Also, I'm always a big fan of inter-disciplinary approaches to information security as I find if you stick with just learning from people who come from the same educational/professional background as yourself, it's very easy to become siloed in the way you look at a problem.

Overall, I think that this entire course should be considered mandatory content for any security management type certifications (CISM, etc) as it provides a fantastically unique view on security that if you're working in info sec management, you really need to understand. 

I'd love to see a follow on, more in-depth course form the same lecturers to go into more detail on the topics covered in this corse, look at some practical examples of analysis and review and compare/contrast the different research that has been published in the area of economics of info sec over the past few years. Hopefully that won't be long coming!

edX: https://www.edx.org/course/economics-cybersecurity-delftx-econsec101x

Book Review: Information Security Analytics

I came across this book on Safari during the week and the title definitely caught my eye as I'm always interested to see new material in the area of data analytics and security. Additionally, when I saw that the book was only around 180 pages, I figured this would be a nice easy read and intro to analytics that I could learn a few bits from and potentially recommend to others.

Unfortunately, the book really disappoints in both aspects and while there are one or two good chapters, I can't recommend reading the whole thing or paying full price for it.

The book starts off really well, giving an overview of what the authors mean by security analytics and mention many of the concepts that are, or can be, used to derive meaning out of data, such as statistics, machine learning, regression, text mining, simulations and clustering.

The second and third chapters give an intro to some of the tools of the trade that are most often used, mentioning R, Python, Simulation Software, Hadoop, Mapreduce and NoSQL databases and then work through some practical examples and exercises around the analysis of log file data, covering loading data into Hive, doing fuzzy searching on the data. What was nice in this chapter is that they use freely available data sets so you can easily download the data and follow along.

The fourth chapter moves on to simulations and unfortunately at this point I felt the book went downhill. Seeing as simulations are a new area for me, I was looking forward to learning something really new in this chapter but I felt that there wasn't enough context put around how simulations could be used for security analytics and while a practical example using anti-virus data is used, it really doesn't flow as well from the previous chapter and appeared a bit contrived.

The fifth chapter then seems to revert back to similar content as the third chapter and covers Access Analytics. However, the structure of this chapter is very strange and starts off with a explanation of what VPNs are, the threats around VPN access and then talks a lot about Python, which seems strange to include here for such a short book and really should have been covered in the second chapter on tools. Unfortunately, this entire chapter effectively is simply an example of using Python and doing haversine analysis, which while really cool and definitely worthwhile, shouldn't really be a full chapter on itself.

I have to say that at this point I was pretty much ready to put the book down but figured I'd continue on seeing as there wasn't much left and I'm definitely glad I did as chapter six provides a really nice introduction to text mining and touches on word association, correlation and clustering. This is a really solid chapter that I'd highly recommend people new to the area read.

The last chapter finishes off with a discussion on the importance of utilizing data more and promotes the reader to go and dig more using the techniques outlined in the book.

Overall, while the book starts off well and has one or two good chapters, I really can't recommend people buy this book or read all the chapters. There's definitely good content in here but the structure, focus and level of detail seems to vary dramatically across each of the chapters and there's overlapping areas that should really have been consolidated.

I really couldn't recommend purchasing it at the current price of around €50 and honestly can't see how the publishers could even justify trying to price the book in this bracket.

I'd really love to see a second edition of this where the material is restructured a bit and made to flow better as I think it could really be a great intro to security analytics if this was done and it was priced closer to the €15 price point.

If you have a Safari or other eBook subscription and can read some of the chapters for free then it's definitely worth a quick flick through, especially to read the first three chapters and the chapter on text mining.

Links:
Amazon http://www.amazon.com/Information-Security-Analytics-Insights-Anomalies/dp/0128002077/
Safari http://techbus.safaribooksonline.com/book/networking/security/9780128002070

Book Review: Designing and Building A Security Operations Centre

When I first came across this book, I really wasn't sure what level of detail or focus the author would be taking so I'll cover that off straight away. While this book touches on some technical areas, this book is very much focused on the organisational, operational and managerial aspects of building and running SoCs.

If you're looking for a book to tell you what tools to select for a SoC, how to architect them, what types of monitoring and response you should be doing then probably best to look elsewhere.

However, if you are interested in working in a SoC, are currently working in a SoC at a junior level or planning to set up or outsource a SoC and new to the area then I'd definitely recommend you read this book.

The first chapter walks through some basics around the types of operations centres and talks through the key phases in developing a SoC, which the author then uses as the basis for the structure of the book.

The second and third chapters discusses SoC customers, event/alert/incident definitions, SLAs and service catalogs before moving onto the various systems and supporting processes that need to be in place to support a SoC, with a particular focus placed on ticketing systems.

I particularly liked chapters four and five, which give good insight into some potential organisational structures and reporting lines for SOCs, high level roles and key considerations in terms responsibilities, breaking it out into options for smaller and larger SoCs. This chapter alone is worth a read if you're new to SoCs as it'll give a good insight into the kind of resources you'll need and ideas on how these might overlap with your existing team.

Chapter six moves into covering the daily operations that should take place within a SoC. It doesn't go into details on incident response but does highlight the importance of root cause analysis and review documentation after the incident has been resolved and the need for communication plans to be in place. It also highlights some of the key challenges in follow the sun models around duplication of technical resources, inefficient handovers and inconsistent training/knowledge sharing and the positives of such models including local data storage for regulatory reasons and reduced local cultural/language barriers.

Chapter seven covers the importance of training and potential approaches to training your SoC team. I really liked the emphasis that the author placed on this aspect of SoC management but I think this chapter could have been incorporated into the previous chapters around teams and people resources.

Chapter eight touches briefly on metrics but rather than dive into long lists of potential metrics, discusses some of the potential approaches to metrics. The section here on vulnerability prioritisation is interesting but for me felt a little out of place within the overall context of the book.

Chapter nine runs through the threat intelligence that will normally be required within a SoC and covers off some of the publicly available resources along with touching on the types of commercial offerings in the area without diving into any particular vendor's commercial offering.

The last few chapters wrap up with some material on outsourcing that is very much worth a read if you're planing on outsource a SoC or engaging with a MSSP. One really nice aspect is that it contains a list of seventy four questions that to consider when selecting an MSSP. I always like to see guidance like this on the selection process for outsourced service as often there's a huge information asymmetry between clients and vendors in info sec and this guidance can at least help clients ask the right questions.

Overall, for the audience I outlined at the start of the review, this books is well structured and a solid introduction to the managerial aspects of SoCs. I felt it could have been shorted down a bit to make it more concise, but other than that it was an informative read and worthwhile for the right audience.

Links:
Amazon http://www.amazon.com/Designing-Building-Security-Operations-Center/dp/0128008997/
Safari http://techbus.safaribooksonline.com/book/networking/security/9780128008997

Book Review: The Frugal CISO

I came across The Frugal CISO purely by accident over Christmas as it popped up as a new addition to Safari and as I'm always on the look out for security books that are a little bit different and non-technical, I figured it would be worth a read.

I'm really happy that I did because the Frugal CISO is a really solid book on both information security management and management in general. This is definitely a book I'd recommend to anyone either taking up a leadership position in an info sec team or setting up a new security team in an organisation. For me personally there were a lot of points that resonated me with as I've spent the last year putting in place a new regional security team from scratch for a large multinational and there's definitely some good lessons that I learned along the way that are included in this book!

I do feel that the title of the book doesn't do it justify however, as while the title seems to indicate that this book will be very focused on cost saving, it's really much more than that. This book is more about being a realistic CISO.. and if there wasn't already a book called the Pragmatic CSO, I'd have called it that!

The book starts off on some very interesting points on the stages of maturity for an info sec team. The approach used differs from the usual maturity levels that we often see in info sec books and focuses more on the organisational aspects of maturity covering things like funding and resource allocation, planning intervals, alignment with business strategy and stability of the team.

The next few chapters of the book focus on giving solid management advice in light of the challenges faced in info sec such as reduced budgets, increasing threats, and competition for staff, with particularly good chapters on team management and hiring that any hiring manager should read.

The book then moves onto a few chapters focusing on alternative views of policy definition and security awareness and a good chapter on taking stock of your info sec program and always considered why you're doing what you do and whether it's still necessary.

The final few chapters have some great advice on budgeting, appreciating that what works in one company, may not work in all companies and on being flexible and aiming to engage rather than block the business.

Overall, this book is full of good insights and ideas for info sec managers and while some of the material will be more general management focused, it's definitely a very highly recommended read for anyone in information security management and one that I'll definitely be recommending. I'd also love to see a follow up book that deep dives a bit more into some of the areas around maturity and engaging with the business as while these are oft discussed topics, they rarely get the level of detail that I think a lot of people would benefit from.


Links:
CRC http://www.crcpress.com/product/isbn/9781482220070
Amazon http://www.amazon.com/The-Frugal-CISO-Innovation-Approaches/dp/1482220075
Safari http://techbus.safaribooksonline.com/book/networking/security/9781482220070