Software, Interrupted

Read all 'Twitter' posts in Software, Interrupted
December 8, 2009 3:43 PM PST

Cloud-scaling on Amazon with Memcached

by Dave Rosenberg
  • Post a comment

One of the headlines that caught my eye today is this blog post from the Amazon Web Services team about a new Memcached as a service offering from Gear6.

Memcached (Credit: Memcached)
For years, Memcached has been used quietly to provide in-memory caching for many popular Web sites, from LiveJournal (for which it was first developed six years ago) to Twitter, Flickr, and Facebook. In the last couple of months, Facebook has opened up about how it scales, and one the key technologies enabling this is Memcached, which services 120 million queries every second. To achieve this, Facebook admits it has had to do some engineering work to improve Memcached's performance and memory efficiency.

Similarly, Gear6 has added features such as replication, clustering, optimized memory utilization and management to create what it calls a Memcached distribution, much in the same manner as Linux distributions are packaged. Joaquin Ruiz, executive vice president of products at Gear6, provided me with additional insight into why Memcached is popular with Web 2.0 sites and why it matters for cloud computing.

The problem, according to Ruiz, is dynamic data services. In a recent blog post, he pointed to the tight connection between dynamic content and Web 2.0; that is, one defines the other. In this Web 2.0 world, the LAMP (and to some extent Java and Ruby) stack "provided a low-cost, efficient development foundation for Web 2.0 but did not free us from the monolithic, vertically oriented, "scale-up" platforms. Memcached provided the heavy lifting in terms of horizontally scaling ("scale-out") on non-monolithic SMP server architectures from Intel and AMD."

In the Facebook example, the Memcached tier stores members' personalized dynamic content, such as status updates, wall posts, etc., so that they can be quickly accessed when queried. It's a similar set up for Twitter tweets or comments on photos on Flickr. While latency in a social application is mildly annoying, latency in a transactional application could mean lost revenue.

Dynamic data services will likely remain an important part of cloud services, which brings us back to the idea of a Memcached service on a cloud platform. Amazon's Jeff Barr noted, "powerful, high-level services like this allow application developers to spend more time focusing on the novel and value-added aspects of their application and less time on the underlying infrastructure."

Anything that developers and companies can take advantage of to serve data faster and more efficiently means they have time to do other things, including increasing their bottom line.

November 4, 2009 8:46 AM PST

Amazon gets social with Twitter integration

by Dave Rosenberg
  • Post a comment

Amazon Twitter integration

Amazon Twitter integration

(Credit: Screenshot-Dave Rosenberg)
Amazon.com this week rolled out an interesting new feature that allows Amazon Associate members to broadcast links to Amazon products via their Twitter accounts.

Amazon Associates is the partner program the company uses as part of its affiliate advertising programs, allowing customers to make money advertising Amazon products.

Associates can now simply click a link in the toolbar to send a link (replete with sales-y text) to Twitter as part of their shopping and selling experience. Amazon gets a sale, Twitter gets traffic, and the associate gets revenue share. What could possibly go wrong?

Linking to Amazon or other online retailers is obviously nothing new, though Amazon has been particularly successful in using its link networks for both sales and to garner higher Google rankings for organic advertising.

This new program does introduce an issue related to link fraud, where spammers and scammers leverage URL-shortening services for spam links. Currently there is no way to verify that the link you click actually goes to Amazon. It's a bit surprising that it decided to use an URL-shortener that it doesn't own, though I suppose the network effect of the URLs helps perpetuate the life of the links.

There is also a risk of nondisclosure wherein in Twitter users attempt to push products that offer some kind of gain to them that they don't clearly state to you. While I understand the argument for disclosure on blogs and media in general, Twitter remains a playground for people to post whatever they want. I highly doubt all the celebrities with accounts would bother wasting their precious time if they weren't posting for their own gain.

Interestingly, there is no mention of whether Twitter is an Amazon Associate, suggesting that Twitter won't see any of the revenue share. I'd like to think that they cut a deal that gives them a piece of the pie, but to date we haven't seen Twitter monetize itself too effectively.

Twitter is quickly becoming the flash news vehicle for everything from news alerts to product placement. And based on a very quick review of my Bitly account, Twitter users just love to click on links. But, I still have to wonder if Twitter will ever get beyond its current role as a marketing tool?

November 3, 2009 12:01 AM PST

Turning Twitter into an application server

by Dave Rosenberg
  • 5 comments

As much as Twitter is a powerful communication and social application, it's a relatively simple Web app. As part of a new contest sponsored by Engine Yard, Ruby on Rails developers are going to turn Twitter into their own application server.

The contest asks developers to program the "Worst App Server Technology Ever" (Waste) using Twitter as the message bus. While much of the contest is being done tongue-in-cheek, it's actually an interesting use case to see if a service like Twitter can take the place of a more traditional message bus like IBM MQ series or AMQP (Advanced Message Queuing Protocol).

Contest participants register up to five Twitter handles and code the function that each would perform in a program. When the contest challenge is issued on November 12, participants will have to use at least 10 of the pre-designated Twitter handles (other than their own) as endpoints to perform functions on data sets located at unique URLs. All messages will work through a series of automated public Twitter replies.

This is somewhere between an application server, a social game, the "telephone game" and service-oriented architecture (SOA) where Twitter plays the role of the enterprise service bus and the Twitter API is the broker between data sources. SOA relies on services exposing their functionality other applications and services can read to understand how to utilize those services. In this case, Twitter can be used as an application server in the cloud. (Take that buzzword bingo players.)

The funny thing is that as absurd and comical as this sounded when the Engine Yard guys told me about it, I've started to think about this as a way to possibly achieve a real technological breakthrough. And while I don't think that Twitter will be the "cloud bus," I do think that there is a lot to be learned from applying this type of constraint to a data flow process.

Engine Yard VP of marketing Michael Mullany told me that the contest shows how developers can leverage a relatively straightforward platform in innovative ways. But it's also another example of an interesting marketing effort to use Twitter as the vehicle for one's own benefit. Also, in true open source fashion, developers wind up building new applications based on code written by their peers.

Let's hope Twitter can handle the attention and developers are not greeted by the ever-lurking fail whale. You can check out the contest and learn more details at Engineyard.com

September 9, 2009 7:38 PM PDT

Avoiding the software 'fail whale'

by Dave Rosenberg
  • 2 comments
Avoid the "fail whale"

Avoid the "fail whale"

(Credit: Twitter)

The tech world is all too familiar with Twitter's "fail whale" and have become accustomed to Gmail failures (which are inevitably chronicled on Twitter.) And while sometimes it's infrastructure (such as routers and switches) rather than software that fails, it often seems as if we too readily accept that software will inevitably breakdown.

Mark Donsky, director of product management at Coverity, commented recently about a recent static analysis of open-source projects performed on the Scan site that showed a 71.9 percent correlation between the number of lines of code and number of defects found.

This is of course, not an open-source problem but a general issue that occurs as more code is integrated into products. I've been told that Windows is developed with two quality assurance people to every engineer as the product has grown over the years.

Coverity is focused on software integrity and advocates static analysis early in the development cycle. While testing of all kinds, including static analysis are obviously good ideas, the tools and methods vary dramatically by engineering organization. The Software Engineering Institute (SEI) at Carnegie Mellon University and the Object Management Group (OMG) recently paired up to form a consortium to establish standards for software quality.

... Read more
August 14, 2009 11:07 AM PDT

Twitter needs to die for microblogging to live

by Dave Rosenberg
  • 6 comments

Twitter has quickly become a cultural phenomenon among the technorati and celebrity set who, for whatever reason, want to share their lives with their followers. And while a new study suggests that the majority of tweets are pointless babble, there is clearly value somewhere or else people would have dropped it months ago.

Twitter

(Credit: Twitter)
That's not to suggest that microblogging is guaranteed to be successful, and certainly not guaranteed to be lucrative, even for Twitter. With no clear revenue model and ever-increasing expectations, the monetization of Twitter remains one of Silicon Valley's favorite conversation topics.

Regardless, the service has introduced a new way to communicate that is unlikely to disappear overnight. As part of the microblogging evolution, Slate's Farhad Manjoo contends that Twitter itself needs to die. And he might be right, at least in the sense that for microblogging to become something bigger media properties and open standards need to find their way into the mix.

If Twitter worked more like e-mail or the Web--a system managed by different entities that were connected by common Web protocols--a hit like last week's wouldn't be crippling. A denial-of-service attack would have brought down some people's status updates, but Twitter would still work for most of the world.

The basic idea is that Twitter-like services should become more like RSS (Really Simple Syndication), an idea which not surprisingly comes from RSS co-creator Dave Winer. And while RSS is not the exact analogy as it's not real-time nor is it geared toward request/response scenarios, it is the model for disseminating information across a vast array of Internet-connected resources.

... Read more
August 6, 2009 4:50 PM PDT

Denial-of-service got Twitter. Is your network next?

by Dave Rosenberg
  • 18 comments

On Thursday, Twitter was taken down by a denial-of-service attack, while Facebook suffered related problems. And other social/media sites like Gawker and Live Journal were hampered by attacks as well. These attacks illustrate just how crucial network security is in a world where organized cyberattacks can bring down even the most prominent sites.

While the news cycle is quickly headed to the point of diminishing returns (lots of ruminations on DDoS, where the attacks originated, and how it was done), I've yet to see posts on how such attacks can be prevented.

In light of Twitter's susceptibility, preventing networks from similar attacks must be on the minds of many organizations. How would you go about protecting your company?

I asked Joe Habib from WildPackets about it, and he provided three tips for preventing network attacks:

  1. Using a network analysis tool, capture all data in one place. All pertinent network traffic can be aggregated to a single location, rather than scattered across the network. Data is captured in a common data format and does not need to be transferred or translated in any way for analysis.
  2. Set up alerts to isolate questionable behavior and zero in on it. If you see requests for considerable more data being requested than normal, look into the matter. Many network analysis tools today allow you to be notified when thresholds are exceeded. You know the typical or average level, so beware of extending beyond the "norm."
  3. Use network forensics data mining tools to reconstruct the sequence of events that occurred at the time of the attack. This will give you a complete picture. If you were not able to prevent a particular attack this time, you will at least gain crucial information to prevent a similar attack in the future.

The right tools help IT personnel get to the root of the problem. Having the appropriate tools in place and following the correct procedures helps eliminate or mitigate the effects of an attack. At least that's what John Pescatore, a security analyst at research firm Gartner, said was the hard lesson that Twitter has learned. "It basically just shows that Twitter wasn't spending the money to filter out DDoS attacks," he said, according to an article in the Los Angeles Times.

Follow me on Twitter @daveofdoom.

July 6, 2009 11:07 AM PDT

Twitter as music marketing tool

by Dave Rosenberg
  • 2 comments
NPD on Twitter and music

Twitter for music sales (click to enlarge image).

(Credit: NPD Group)

Recent research from NPD Group says that Twitter users are more engaged with music, both online and offline, than their non-tweeting counterparts and also are more likely to spend money on downloaded music.

This data obviously suggests that Twitter could be a valuable tool to music marketers, currently struggling to figure out the best ways to monetize music (as opposed to the existing strategy of suing listeners.) But it's hard to see a direct financial benefit until Twitter usage can more directly correlate to sales, including sales that are influenced by followers etc.

People on Twitter purchased 77 percent more digital downloads, on average, than those who were not tweeting. One-third listened to music on a social-networking site, 41 percent listened to online radio (compared with 22 percent of all Web users), and 39 percent watched a music video online (versus 25 percent of all Web users).

"NPD's latest music-acquisition study shows that there are segments of consumers who are more actively integrating Twitter as a key tool for communicating and networking," Russ Crupnick, entertainment industry analyst for NPD, said in a statement last month. "Based on their music-purchasing history, active Twitter users are simply worth more to record labels and music retailers than those who are not using Twitter."

Crupnick also noted in the statement that Twitter has the potential to help foster the discovery of new music, and improve targeted marketing of music to groups of highly-involved consumers, but only if used cautiously and judiciously. "There must be a careful balance struck between entertainment and direct conversation on one hand, and marketing on the other," he said.

The underlying theme in this report is that Twitter users are slightly more connected--or perhaps more "always on" the Internet, and accordingly consume more music online. The disparity in data isn't dramatic enough to suggest that Twitter users are completely technocrats, but it does suggest that there are low-cost ways to reach music consumers--and to get them to pay for music, something the major labels can't seem to figure out on their own.

Follow me on Twitter @daveofdoom.

June 9, 2009 11:32 AM PDT

Twitter and blogs: Post once and bail out

by Dave Rosenberg
  • 3 comments

For all of its glory, Twitter is apparently not as sticky as many social media buffs would like it to be. A recent Harvard Business School study reported that 10 percent of the service's users account for more than 90 percent of tweets. (I wrote about Twitter's lack of loyalty back in April.)

However, I don't think it really matters. As with any service or piece of software, a rising tide lifts all boats, so a core user base can propel a service for quite a while. Somewhere down the line however, Twitter as a company will need to put programs and efforts into place to encourage people to actually use the service if it ever plans to monetize it.

The fact that 10 percent of users are driving 90 percent of the content is not dramatically different than what you see with sites like Wikipedia, or with personal blogs, which have an even lower rate of consistent publishing. According to a 2008 study by Technorati, 95 percent of the blogs they track hadn't been updated in at least four months.

Orphaned tweets, like orphaned blogs, are just as much part of the social fabric as anything else. The fact is that people abandon stuff all the time--TV shows, books, whatever. We shouldn't be remotely shocked that someone bails out of blogging or something else that could be considered work.

... Read more
April 28, 2009 1:57 PM PDT

Twitter's lack of loyalty--an Achilles' heel?

by Dave Rosenberg
  • 19 comments

Some interesting data from Nielsen suggests that Twitter, despite the hype and meteoric growth, appears to have a user loyalty problem, an issue not suffered by Facebook or MySpace, the two behemoths of social networking.

Considering the viral nature of Twitter, I was a bit surprised to see that users weren't more loyal. On the other hand, sites like Facebook and MySpace offer a lot more functions that facilitate communication on many levels, not just through messaging.

Currently, more than 60 percent of Twitter users fail to return the following month, or in other words, Twitter's audience retention rate, or the percentage of a given month's users who come back the following month, is currently about 40 percent. For most of the past 12 months, pre-Oprah, Twitter has languished below 30 percent retention.

Is this Twitter's Achilles' heel, where there aren't enough reasons to come back to the service, or, is audience retention naturally lower because Twitter only offers one thing to do in comparison to the other services?

The chart below shows that MySpace and Facebook both had higher loyalty levels at the same reach.

Twitter lacks loyalty

Twitter lacks loyalty

(Credit: Nielsen Wire)

I think it's an easy argument in Twitter's favor that it has significant reach with significantly fewer features than the comparative sites. Of course, loyalty matters a great deal if and when the company ever decides to monetize the user base.

April 21, 2009 8:51 AM PDT

Things to know about social-media marketing

by Dave Rosenberg
  • 1 comment

Social media programs have become legitimate marketing tools, providing immediacy and insight into communications. But, they require a great deal of attention to maintain and a fairly sophisticated level of metric analysis to ensure that the programs are valuable.

Creating corporate blogs and Twitter accounts or developing Facebook inclusion programs are just a few possibilities, and there are many more options for marketing teams looking for new bang from their PR dollars.

Over at the PageOne PR Page Wonders blog, Jenna Boller details Five Things You Need to Know About Social Media Marketing and PR

  1. Social media requires a willingness and readiness to engage.
  2. Your audience might not naturally pay attention.
  3. Social media never stops.
  4. Social media requires A LOT of planning.
  5. Social media is everyone's responsibility.

Social media is becoming part of every company's marketing strategy. There is a great deal of measurable value from these tools provided you have established a baseline of what you hope to achieve.

Disclosure: I occasionally borrow office space from Page One PR.

Follow me on Twitter @daveofdoom

advertisement

About Software, Interrupted

In "Software, Interrupted," Dave Rosenberg discusses disruption in the software market, as well as the products and services that keep business technology norms in perpetual flux.

With nearly 15 years of technology and marketing experience spanning from Bell Labs to multiple start-up IPOs, Dave co-founded open-source software company MuleSource and now serves as general manager of Hardy Way. He also happens to be a U.S. patent holder and a workaholic. Technology is his best friend and mortal enemy.

Add this feed to your online news reader

Software, Interrupted topics

Most Discussed