Next week, the team I am on at Automattic is meeting up in Tel-Aviv to attend the NetSci X conference. There is so much to be excited about — the opportunity to spend 10 days with colleagues, the interesting talks on network theory and analysis, the opportunity to visit the holy sites of the Bahá’í faith and soak up the spiritual energy, the gift of spending days in some of humanity’s most ancient cities, and then the magic of the unexpected.
I am looking forward to experiencing and blogging about those adventures. I want to take time to discuss one of my favorite things to do on long plane rides — reading books that I have long had on my list and rarely get the chance to dive into.
There is a Lonely Planet travel guide to Israel (maybe better for pre-travel reading? My colleague has hooked it up!), the memoir Seven Good Years by Etgar Keret who I’d enjoyed listening to on Fresh Air, Blood at the Root an analysis of one Georgia town’s painfully recent struggle with its history of racism, The Attention Merchants on the impacts of the behavioral advertising models that dominate the Internet, and Weapons of Math Destruction which I’d started but not finished. I have laid them out on the table tonight and thinking today about which one or two to take with me.
Usually it happens that a fellow traveler feels compelled to share something and I am all there to listen. On a journey back from Vancouver, a fellow traveller shared beautiful and haunting pictures she and her husband made of the botanical gardens around Vancouver.
I want to be open to all that can happen on this journey. But it is great to have books on the plane to share the ride.
So I thought I would take the inaugural post on charlesearl.blog to commemorate the great dearly departed Tupac while also dipping my foot into Deep Learning Neural Style Transfer. What??
Honoring you with style
Style transfer[1] is the AI that powers apps like Pikazo and Prisma — it extracts the style in from one image then re-images a second image in that style to create a third novel image.
When a deep neural network is trained to classify images, something really interesting happens. During the training process, the network is discovering features that can be used to accurately identify images — millions of images might be analyzed. The feature-specific knowledge gets organized in a hierarchy. This knowledge is split across the layers of the network roughly according to complexity — the lower level layers tend to capture relations at the pixel level whereas the units in higher levels of the network capture the relations between abstract entities in the image.
In the neural original neural style paper [1] the authors use a pre-trained deep neural network in kind of the same way that a doctor uses an X-ray machine. That is, as an instrument that can discover structure not immediately apparent. While the doctor can get some idea of the condition of your bones by careful use of the X-ray source, the researchers use the network to extract both “style” and “abstract content” from the artwork and the photograph. In style transfer, spatial correlations between image features — explicitly defined by the Gram matrix — are used as a proxy for style. Cleverly reconstructed images corresponding to the output of each convolutional layer of the network are used as a proxy for abstract content.
Constructing the composite image then is a matter of overlaying the style (Gram matrix representation ) and content images for selected layers. Through some experimentation, the authors of the original algorithm [1] arrived a set of layers that seem to work best for style and content respectively. This is still an active area of experimentation. Prisma uses clever custom algorithms to speed the stylization process so those hip images can appear in your Facebook post in reasonable time.
After experimenting with a few implementations, I made some headway (given quirkiness with my NVIDIA 1060 graphics card and the various deep learning libraries ) with an implementation of neural style transfer by Justin Johnson at https://github.com/jcjohnson/fast-neural-style. It is further described in his paper[2].
Now on to the image making! Given that Tupac’s mother Afeni Shakur also passed away this year (her memorial in the New York Times is a stirring testament) I thought it would be appropriate to look at the art of the 1970’s Black Power movement as a source of style. The painting Revolutionary by Wadsworth Jarrell seems to capture the urgency of the lives that Afeni and Tupac lived.
What an image! Both the rich structure of the image and the fact that I hadn’t come across many examples of the Black Arts movement work in style transfer seemed to make this especially gratifying project.
So here’s what I came up with.
On the left are the original photos of Afeni Shakur (taken in the ’70s) and Tupac at the height of his art. On the right Revolutionary. The new images are in the center. To my naive eyes, the new image of Afeni works a bit better. Sorry Tupac. But the other thing that has me intrigued is the textures in Revolutionary. Jarrell has embedded a lot of symbolism: you can pick out “Black”, “Revolution”, “Unrest”, and “Black is Beautiful” in the painting. Are these symbolic elements recognized at any layer in the network? There are clearly some interesting questions at the intersection of art, computer vision, and deep learning.
To end my post, I’m going to leave with an experiment based on Romare Bearden‘s Train Whistle Blues. I think this works a little better for Tupac.
Now I’m really interested in seeing how clever we can get with identifying interesting textures for the transfer.
I find that Oleg Kiselyov’s blog is a garden of delights for those of us who enjoy Lisp and Scheme. Each post seems like a timeless treasure on things to really know and appreciate about functional programming — maybe programming generally. I am still digesting his Monadic Programming in Scheme because I am still trying to grok Haskell.
As a kind of challenge for really taking his lessons to heart and mind, I wanted to assemble a list of things to write in scheme, perhaps with an eye toward thinking through modern type theory. Here is a beginning to the list
min-hash. A year or so ago was impressed that you can get very accurate duplicate detection with simple n-gram hash matching.
keras. Or rather a high-level language for composing deep neural networks. The keras functional interface still doesn’t seem quite functional to me.
automatic differentiation. The post Neural Networks, Types, and Functional Programming raises the interesting parallels between deep learning and functional programming. On looking further I discovered that there is more than one way to do backward differentiation. That is, there may be some useful efficiencies that both the machine learning and automatic differentiation communities might jointly discover.
So there’s a 15 minute start to the list! I’ll keep adding as I dive into things. In the meantime, there’s a lot in Kiselyov’s blog to catch up on.
It seems that my last post set sparked a lot of questions in my mind about the Civil War era, it’s impact on my family, the repercussions upon Georgia, and how it is still being grappled with to this day.
There is a lot that I am still unpacking but I’ll start simple. For some reason I have been fascinated by Ulysses S. Grant since since sometime in grade school: at some point in the fifth grade I had written a play about a family of newly freed slaves encountering him on their way to a new existence in the north.
I encountered a biography, Grant by Jean Edward Smith which was fascinating in laying out many of the details of the Civil War I’d forgotten. He has a great story-telling style that seemed to make the intrigue of the early days of the Civil War jump out, and brought to life the way in which life in the 19th century seemed to straddle the complexities of a long ago time with our own.
I read a few chapters of it before returning it to the library, and then shortly a long awaited biography American Ulysses came out. Prescient. I vowed to complete this one end to end. As the politics of the 2016 US presidential election came to their head, I was reminded time and again, how far and how little progress the US has made confronting what I would simply call White supremacy. Several insights began to stand out as I continued reading the book.
One conclusion that one comes to is that the relative low ranking of Grant in terms of presidents seems in part due to the revisionist US history that unfolded after Reconstruction was abandoned. That is, objectively Grant stands out as an equal to LBJ and his predecessors as an aggressive advocate of civil and human rights. Yet after his time in office, the country effectively began to turn a blind eye to the racial violence and dis-enfranchisement of African Americans ( and Native Americans, and ethnic minorities ) that would envelop the South for a hundred years. I think the historians bought into this narrative.
The degree to which he evolved as a thinker on race and equality seems unparalleled in politics. These kinds of evolutions are certainly rare in US politics, the closest thing that I can think of in this evolution would be Robert Kennedy’s evolution from a critic of the 1960’s Civil Rights in his time as Attorney General to being an unqualified advocate at the time of his assassination.
Andrew Johnson’s impeachment seems to have really arisen over his refusal to enforce any of the Civil Rights measures passed in the immediate aftermath of the Civil War. This biography argues effectively that (Andrew) Johnson didn’t have any interest in African Americans having voting rights . To quote Johnson “to grant the privileges of citizenship to blacks would show prejudice against whites.” We still seem to be having the same zero sum game issues in 2016.
It is amazing to note his development of and unflinching support of the 14th and 15th amendments. To quote: “I will not hesitate to exhaust the powers thus vest for the purpose of securing to all citizens of the United States the peaceful enjoyment of the rights guaranteed to them by the Constitution and laws.”
Grant’s development of a real relationship with Mexico in dealings with Benito Juarez and Matias Romero — again remarkable for the “manifest destiny” era — again have lessons for today. I’m having a hard time conceptualizing how Grant would react to a fellow American Republican president building a wall to separate the two countries.
In sum the book was an odyssey of hope. The journey of a man who evolved from being indifferent to slavery to one of the architects of a multi-racial society is dramatic and pulled off with great story telling on White‘s part. Especially insightful and compelling are White’s narrative of the how Grant confronted the Ku Klux Klan. The gem of this story is how Grant pulled in an ex-Confederate from Georgia, Amos Akerman, who would champion civil rights as U.S. attorney general and as a lawyer in Georgia when he left the cabinet.
There is a quote, though antiquated, that White pulls out that seems to sum up what a real politics could be “Treat the negro as a citizen and a voter, as he is and must remain, and soon parties will be divided, not on the color line, but on principle. Then we shall have no complaint of sectional interference.” In essence when we are all free to express our voice, not threatened with removal or invisibility by our identity but included by our humanness, then governance can enter a civil space.
The way the book dealt with the fine details as well as the broad historical context is amazing. I appreciated his understanding of Grant as an introvert who was able to provide critical leadership to a country divided.
Finally it seems to me that it is another testament to Dr King’s quote: “The arc of the moral universe is long, but it bends towards justice.” When I first heard that quote, I’d no idea that it would take so long. I am learning patience.
Six years ago, I was trying to get a proposal off and my wife hooked me up with a few days of retreat time at a place called Banning Mills. It’s about an hour and half from Atlanta, but in many ways, rural Georgia has always seemed a world away.
Both of my grandfathers left small towns in the southeast — Covington, GA and Salem, AL — under a phenomenon I’ll just call “Klan duress”. Veiled and not-so veiled threats from powerful White men led them both to migrate to Atlanta before the 1920’s. So I acknowledge that I still have some biases to overcome.
Nevertheless, I was impressed by the beauty of the surrounding area. It claims a famous zip-line
and nearby there are exquisite views of the Chattahoochee that are priceless for their serenity
The image that remains with me to this day is a placard that I saw on entering the resort’s main building to check in. It is of a Black man, a very distinguished gentleman, taken in the 1850’s/60’s. I was shocked by it’s very existence — that in Carrol County there was Black who then and to this day occupies a position of honor.
The gentleman was Horace King, an Oberlin-educated architect. He built many bridges throughout the southeast, in fact a bridge connecting Alabama and Georgia. Before the war broke out he had purchased his freedom using monies from his bridge building efforts with his owner John Godwin. He is responsible for this innovative fantastic staircase in the Alabama State Capital, was conscripted into building ships and defenses for the Confederacy, and was an active figure in the rebuilding of Alabama after the Civil War.
It was no small feat. According to the census data ( the visualizations developed by the Census Department during the pre-Tableau 1860’s is in itself a feat of imagination and an exemplar of good data design ), more than 40% of Georgia was enslaved at the time.
Though the “invisibility” of the my African slave ancestors is acknowledged, the census data paints a picture of a similarly un-empowered White population getting by on subsistence farming.
The image I left Banning Mills with was of Mr King standing with the White, Black, and Native American bridge builders near a completed project. I’ve probably imagined that image — so far I’ve not been able to locate it despite generous help from Banning Mill’s owner and founder Donna King. But I believe the impact of his life captures that impression: that he created bridges in his lifetime and today that allow us to escape the received and limited narratives that constrain our ability to connect, to see, and to grow.
I ordered a Kinesis keyboard on joining Automattic, figuring that I would try to kick the new year off taking ergonomics seriously. It feels like I have so far gone through two periods of learning, and still getting with respect to matching the rate of touch typing on Mac laptop keyboard.
The first phase was rough, feeling like a basic hunt and peck. I’d quickly get frustrated and go back to the laptop keyboard, or the small Mac wireless silver keyboard. So somewhere between one and two hours a day. That phase was from late February until say around first week in March. I took a couple weeks break out of frustration and then picked up last week in March.
Phase II seemed remarkably better, much faster touch typing to the point of being relaxed and less having to think about where the keys are. The challenge so far seems to be around getting into the zone as far as emacs keys are concerned. That is, the META- and CONTROL keys still present a problem for me, and it still seems a bit weird in terms of the Mac key mapping for the kinesis (I’m using Advantage ). Having the the CONTROL key be right next to either thumb seems like the more appropriate mapping to have. I still haven’t quite figured out how to deal with numbers effectively. Strangely enough, it almost seems like although I’m creating globs of floats everyday, most of those are generated by the machine.
The real issue is having faster access to the numeric operators, parens, brackets. So maybe the next goal is to work on those python and scala typing drills eh?
I suspect that Clojure, event though acknowledge to be the latest incarnation of the common lisp family of languages, is less powerful than Common Lisp. I suspect that this has a lot to do with macro processing, the fact that it is a JVM language.
Is it worth pursuing to any degree? How would one go about firming such a claim, and to whom would it matter?
#/media/File:Dillingham-street-bridge-1900.jpg){kind=link}
#/media/File:Alabama-State-Capitol-spiral-staircase.jpg){kind=link}
#/media/File:CSS_Muscogee.jpg){kind=link}