This week I’ve attended the Network Automation Seminar organized by Reiss Romoli. The speaker was the great Ivan Pepelnjiak! I was happy to meet Ivan again after NFD16…

Also possibly the shortest bio on the podcast…

I’ve never had a woman tell me she was assaulted by anyone either, but given my past behavior, even if someone I knew had, I don’t think my actions made me someone that a woman would trust to tell. Let me give you some examples of my behavior. Just so we are clear, none of this got me any dates in grade school…

Learning the hard way About a month ago David Robinson made a tweet that I both agree and disagree…

Why CircleCI? Yes, I know using Travis CI is this easy, thanks to devtools package: Travis CI is OK most of the time. Still, CircleCI has some advantages: Though Travis can cache the setup once it succeeds, it is good if we can save time to setup testing environment by using existing Docker images. CircleCI displays the test summary in this pretty way: All I had to do was two steps…

Day two of NFD16 with Cisco. The presentation was split in two parts. First part for Intersight, second part for Tetration. I’ll post here just a few thoughts about Intersight…

One of the concepts introduced in the book is stock and flow diagrams that represent basic systems. The first diagram in the book shows a basic stock with an inflow and outflow…

IN THIS POST I WANT TO SHARE SOME CODE TO CREATE AN ANIMATED CHART of housing starts…

Hello everyones, As the french health insurance is a public institution, it may be more interesting to find a way to monitore data than finding a way to refund less drugs… Hence, it may not be readable to show the 84 categories, so I have decided to select just some of them…

This post is the third part in a series of blog posts on Signal Detection models: In the first part, I described how to estimate the equal variance Gaussian SDT (EVSDT) model for a single participant, using Bayesian (generalized linear and nonlinear) modeling…

10月上旬にBostonに出張に行ってきました…

10月上旬にBostonに出張に行ってきました…

Within a cut-point region, the sum of the basis functions always equals 1. This is easy to see by looking at a plot of basis functions, several of which are provided below. The definition and shape of the basis functions do not in any way depend on the data, only on the degree and cut-points…

This is going to be a short post. Because the only thing I did for this package was to blatantly copy the work of Colin Morris on song lyrics self-similarity matrices (a.k.a “SongSim”), into R…

A clever friend of mine says: the main programming language, is not Java or Python but English. So, when you’re searching for help, you’ll be much successful if searching in english…

When building this site, the two resources I consulted were: It might seem complicated, but with some coffee and tenacity whole thing can be done in an…

IN THIS POST I WANT TO REVISIT FLEXDASHBOARDS. Back in January we made several Flexdashboards with R to display economic data. See my guide to building a flexdashboard for some examples…

Earlier this month I did a post on simulated annealing, an algorithm that I learned in the class I’m currently taking: Advanced Statistical Computing taught by Chris Fonnesbeck here at Vanderbilt. Recently we have moved on to more traditionally “statistics” algorithms, and one that Dr. Fonnesbeck is particularly well versed in, Markov-Chain Monte-Carlo (or MCMC).…

The resulting map is rather sparse and does not provide much information, but it gives a good starting point from which to build a more informative map…

This post is the second part of a series of three blog posts: In the first part, I described how to estimate the equal variance Gaussian SDT (EVSDT) model for a single participant, using Bayesian (generalized linear and nonlinear) modeling techniques…

We’ve been hard at work for 18 months on a new processing engine called Wallaroo for deploying and operating big data, fast data, and machine learning applications…

Synopsis Market Research is great at compiling the right data, but not so good at making it easy to use. This isn’t about “storytelling”. It’s about the data itself, and clarity on what delivering it actually means…

Available discrete distributions: Available continuous…

I HAVE BEEN EXPERIMENTING WITH A NEW WAY TO VISUALIZE DATA, a bivariate tilegridmap. When I get around to rolling out my tidyPowerPoint workflow we’re going to want something other than bars and lines to fill it up…

Theory to Application 1 I recently realized my review of Michael Alvarez’s edited volume Computational Social Science: Discovery and Prediction went online a while ago…

The story of my first tweetbot, complete with forays into Azure and Markov…

Andrew is the founder of 3wks who have delivered 190 projects on Google Cloud platform for enterprise customers in Australia. He loves everything serverless, from App Engine through to BigQuery. Graham is a senior software engineer based out of Melbourne Australia, and works for Shine Solutions…

Rstudio has recently launched a community site and it seems to be providing a great forum…

WHILE I WAS TRYING TO MAKE TIME FOR TIBBLETIME yesterday I got distracted and made this plot: hey! here's a #dataviz: tile plot of U.S. state unemployment pic.twitter.com/vH8fSu63ve - Leonard Kiefer (@lenkiefer) October 10, 2017 In this post, let’s go over how to make this plot with R…

This post is the first part of a series of three blog posts: In the second part, I describe how to estimate the equal variance Gaussian SDT model for multiple participants simultaneously, using hierarchical Bayesian models…

and then load the data with: First let’s select a list of countries: As I said before, if you do not re-order the countries inside the data frame, the names of the files and the plots will not match…

WHAT TIME IS IT? TIME FOR TIBBLETIME! In this post, I’m going to take the tibbletime package out for a spin. Turns out this package is quite useful for things I tend to do. We’ll use the tibbletime package to write some R code to extend our ongoing analysis of trends in the U.S…

TIME FOR ANOTHER DATA WRANGLING AND VISUALIZATION EXTRAVAGANZA. This time we are going to work hard to turn some big data into little data. That is, we’re going to work hard to aggregate several million loan level records into useful summary graphics to tell us about the U.S…

For my very first course in Statistics during my undergrad years, we had to make “a project”, defined as “go out there, gather some data, summarize them and perform proper statistics on them”…

Automation is today a word that can be both scaring and exciting at the same time. The scary part is often related to the question: will robots steal my job? Federico Pistono says that’s ok…

From being an avid daytime moviehouse attendee…

This is an excerpt from a talk I gave at LASTConf Melbourne on Web Performance Optimisation…

Suppose we have a data that has too many series like this: For such data, it is almost impossible to identify a series by its colour as their differences are so subtle. So, I am motivated to filter data and map colour only on that, using dplyr: (Does “non-logical predicate” make sense..…

As part of the release, I thought I’d explore the negative binomial just a bit, particularly as it relates to the Poisson distribution. The Poisson distribution is a discrete (integer) distribution of outcomes of non-negative values that is often used to describe count outcomes…

I AM HEADED OUT TO ARIZONA to talk with mortgage professionals. I wanted to share some charts I’ve put together for the Arizona and Phoenix metro economy. These charts were put together using R and tidyquant as I described here…

Introduction Distributed systems are hard…

I AM WORKING ON ADDING some more analysis around mortgage origination trends (see here for a high level summary). It’s on the way, but let me just leave a few graphs for you. These are updated versions of the same ones we made last year…

I read this post on Ivan’s blog this morning and as often happens it made me think. The key point is: figure out what your most pressing problem is. Who should do that? Let’s analyze the roles involved in a typical SMB or enterprise…

Game of the week WBA 2 - 2 WatfordThese teams probably won’t feature much but Watford deservedly fought back from a two goal deficit Widening gap in classIt was generally forecast that the top-six of last season would repeat again in this campaign - though probably not in the same…

Introduction Authentication is one of the features the open source version of shiny-server is missing. The simplest way is to set up a proxy and let it handle the user authentication. But in some scenarios, this isn’t sufficient as you are not able to determine in shiny who is the current user and thus are e.g…

A Dive Into Some Global Flooding Data I always like to keep a look out for interesting open data sets. One great resource for such things is Jeremy Singer-Vine’s Data is Plural weekly newsletter that brings together a collection of “useful, curious datasets” for us all to enjoy and wrangle with…

I’ve BEEN MESSING AROUND MORE WITH R and OFFICER and having too much fun for a Monday. I’m going to dive into some details later, but I’ll just leave a couple files here. See the attached PowerPoint .pptx file for all the charts…

Fill in the relevant information for your account. The team is usually your username, but it can be different…

We can save the objects for later use by using the assignment operator and giving the data frames names. Now that we have the data in a better form, we can plot it by making another bar chart…

Meme Me Up, Scotty! Every now and then on Twitter or Facebook there’s a math problem posted which challenges users to get the right answer. They’re almost always based on order of operations and the reason most people get them wrong is because they don’t remember that elementary rule…

This post will be dedicated to template references in Angular and dynamic scoping. Although the main focus of the article will be understanding TemplateRef we will also touch semantics of programming languages by introducing the concept of scope of name binding…

Anyways, given that more waves are available, we don’t have to look at a single comparison between 2007 and 2016, if we want to push Bruenig’s work a little further…

Welcome to my personal website. This site will serve a a landing page for my public projects in development…

Awesome Blogdown Collage by Mara Averick Earlier this year, at RStudio::conf 2017, I heard about Yihui Xie’s Blogdown package for the first time…

IT IS SEPTEMBER AND THAT MEANS it is data release season. One of the most important September data releases for me is the annual HMDA data release. These data provide the closest thing to a publicly-available comprehensive summary of U.S…

Day two of NFD16 started with Apstra and their intent-based networking system. Intent concept is not as broad as SDN but still vendors have different views of this meaning. According to Apstra an intent is “the definition of the expected outcome”. The sum of the intents of a network is the source of truth…

NB This is a revamp of a previous post The World Bank is a great source of economic and social data…

I am generating multiple data sets and estimating the causal effects for each using the EM and IV approaches…

I always love discussions about R release names and their origin…

Avere provides high performance file system solutions for hybrid cloud infrastructures, and is partner for Google Cloud Platform…

Game of the week West Ham 2-3 TottenhamThe star of this week’s post, Christian Eriksen, gets involved in all three goals for Spurs After six games, Manchester Utd…

I’m very excited to announce the first open source, public release of our ultrafast and elastic data processing engine, Wallaroo…

IN THIS POST I WANT TO GO THROUGH SOME EXAMPLES of using the purrr package for R. Now there are already some great examples of how to use purrr…

Motivation Why do we need dummy variables? Ways to create dummy variables in R stats package dummies package dummy package caret package Performance comparison Smaller datasets Large datasets Conclusion Qs Motivation Very recently, at work, we got into a discussion about creation of dummy variables in R…

NEW HOME SALES FALL according to the latest new residential sales report from the U.S. Census Bureau and Department of Housing and Urban Development (HUD). errr probably. Remember, housing data is uncertain and there’s quite a large margin of error. Per the Census/HUD report sales fell 3…

Sometimes a small change in approach can make a big impact on the outcome. Where we were A large client of ours had recently started their cloud migration journey. I was brought in as an iteration manager to help one of the recently renamed Cloud Foundation squads manage their work…

This is great for long lists, since you can return multiple item simultaneously without needing to endlessly click items in a list! The shiny module works with two main…

I have been trying to decipher for myself, what is in the current (well, yesterday’s) Graham-Cassidy health care bill…

IN THIS POST I WANT TO SHARE SOME R CODE to create charts of U.S. housing starts we studied last week. Get data We’ll use tidyquant (see e.g. this post for more) to go get our data from the St. Louis Federal Reserve Economic Database (FRED). We’ll also use cowplot to arrange multiple ggplot2 graphs on one page…

A recent paper, Redefine Statistical Significance by 72 (😱) co-authors, has caused quite a stir in the statistical community…

TLDR: I made a thing that moves and has sliders and stuff. It’s down here. Currently I am taking a class titled “Advanced Statistical Computing” taught here at Vanderbilt by Chris Fonnesbeck. The class is a fantastic whirlwind tour so far of some common optimization algorithms used in statistical computing…

This post details some tips and tricks for writing a thesis/dissertation using the bookdown R package by Yihui Xie. The idea of this post is to supplement the fantastic book that Xie has written about bookdown, which can be found here…

As a father of three girls, Facebook decided recently to show me this video, titled “If you have a daughter, you need to see this”: It shows a mother and daughter gradually removing books from a children’s books bookshelf in a typical book store…

LOOK I DON’T HAVE ANYTHING BAD TO SAY about PowerPoint. Others have said it (see for example Tufte and Harvard Business Review). It’s a tool and a fact of life for many of us. I am interested in making better PowerPoints…

Second part of Tech Field Day NFD16 day 1 continues with Arista. Getting ready for @AristaNetworks #NFD16 pic.twitter.com/SCWXVGsR2l - Gian Paolo (@gp_ifconfig) September 13, 2017 Arista’s presentation included 400G (hist: it’s fast!), EOS programmability, Network Automation and Telemetry, Routing Architecture Transformations…

You have a data frame you want to alter using piping, something like this. As far as bugs go, this is fairly inconsequential…

THIS WEEK WAS BUSY, with a lot of data releases. Earlier this week we talked about housing starts, but there were a bunch of other key releases. Let’s review some of them here. We’ll just do a quick description of the data and then follow up with a static chart and an animated gif…

TLDR: Little’s MCAR test is unable to tell data that are MCAR from data that are MAR in small samples, but maintains the nominal error rate when null is true across a wide range of sample sizes…

TLDR: When performing a simple linear regression, if you have any concern about outliers or heterosedasticity, consider the Theil-Sen estimator. A simple linear regression estimator that is not commonly used or taught in the social sciences is the Theil-Sen estimator…

It is probably easiest to see in action: Here is what the data look like: Here is a slow-motion version of the EM estimation process. I show the parameter estimates (visually) at the early stages of estimation, checking in after every three steps…

The ggplot2 package and its extensions dominate the R visualization landscape, particularly for static charts It got me thinking about how rare was that combination of two ’g’s to start a word - at least in the English language…

First day at Tech Field Day NFD16, I’m quite excited to be here among fellow network engineers to share our views on products and technologies. I’m planning to post my takeaways for each vendor with variable lengths based on my knowledge and interest of the specific product…

His long-term goal is to build the products and infrastructure to support the future world where billions of people have coding fluency (just as billions of people today can read and write)…

Game of the week Watford 0-6 Manchester CitySecond time Man City have won by 5+ goals in consecutive games against same team…

ARE HOUSING STARTS GRINDING HIGHER, OR GRINDING TO A HALT? Today the U.S. Census Bureau joint with the U.S…

På de respektive facebook-sider kan vi naturligvis også finde antallet af følgere - dem gemmer vi også lige, da vi skal bruge dem senere…

TLDR: In small samples, the wild bootstrap implemented in the R hcci package is a good bet when heteroskedasticity is a concern. Today while teaching the multiple regression lab, I showed the class the standardized residuals versus standardized predictor plot SPSS lets you produce. It is the plot we typically use to assess homoskedasticity…

LET’S WRANGLE SOME HOUSING DATA. We’ll try something different with how posts are organized. In the past I have generally mixed data wrangling, R code and graphs all in one post. Now I’m going to break it up. Posts like yesterday will just show some data and discuss it…

I’m currently working on a small part of a project for my new compagny involving VRT, Geoserver and Tomcat on a CentOS VM. Easy, you should say, not to easy when it is your first time with all those components (except for GeoServer that I tried already). So everything was new to me…

Introduction In cricket, a Manhattan chart records how many runs a team score in each over (and how many wickets fall)…

While working on a couple of projects involving beer data I did land up at breweries association. They have a data set on all the breweries across the US. This raw html table was scrapped ,cleaned and geocoded using R. You can find the R Scripts which I used for this project here…

LET US REVIEW SOME INTERESTING TRENDS IN HOUSING VACANCIES for the United States. Earlier this year we talked about how limited housing supply was helping to drive accelerating house prices across the country. In such an environment you would expect to see housing vacancies decline…

The motivation An example using BostonHousing data Load libs & data Create a starter dataframe Select the models Create data-model combinations Solve the models Extract results In conclusion tl;dr: You’ll learn how to use purrr, caret and list-cols to quickly create hundreds of dataset + model combinations, store data & model objects neatly in one tibble, and post process…

Hello everyone, Now it should work! You can find the whole process in this repository. We started with something simple but you can do more complex things by applyng the same logic…

ALL RIGHT! LOOKS LIKE WE’RE BACK ONLINE. I took the opportunity to try out the newly released blogdown R package and migrate my blog over from Jekyll to Hugo. This blog has been up for just over two years and Jekyll was working fine, but I never felt really comfortable…

I already wrote a couple of essays related to the development of programming languages that I was extremely excited about! For instance, in “Static Code Analysis of Angular 2 and TypeScript Projects”[1] I explored the basics of the front end of the compilers, explaining the phases of lexical analysis, syntax analysis and abstract-syntax…

In this post, I will focus on spark histograms in summary statistics and beyond…

Tino hails from the BigQuery team, where he solved difficult cloud-native product problems, enabled Googlers and customers, and built programs like BigQuery Pacific…

A few weeks ago I did a post on the StatCan API I have since come across the new cancensus package, a wrapper function for CensusMapper API, in beta and not yet available on CRAN . These has been developed by the Vancouver team at MountainMath featuring Jens von Bergmann…

Here’s the problem…