First, the article suggests that the NIST competition to choose a new cryptographic hash algorithm to be standardized as SHA-3 has not concluded.  NIST selected KECCAK as the basis for SHA-3 last year.   KECCAK has many applications and it will be a while before the its application in an SHA-3 will be standardized (in an anticipated FIPS 180-5).  See the 2013-02-06 presentation, “Keccak and the SHA-3 standardization.”

There’s also a remark concerning the ability to attack PBKDF2 easily.  PBKDF2 is an iterative, salted key-transformation procedure that can be based on a chosen PRF for the transformation.  It is typical for PBKDF2  to conduct a large number of HMAC-SHA1 iterations.  Other MACs can be used.  The idea is to raise the work factor to make brute force attack on the password infeasible.  The problem with PBKDF2-derived hashes as a password authentication approach is not the ease of attack but the imposition of that work factor on server-side authentication procedures.

So long as the password-based authentication procedure is required to be efficient and economical, the problem is not the hash.  The problem  is the password choice and the poor security that allows the hash to become known.   Once the hash is disclosed, adversaries have the same efficiency advantage, and all the time and resources they need, to discover the password.    Improving the hash does little to mitigate this serious problem.   New password protocols are required.

I’ve gathered my considerations on what is required to confine the consequences of hash disclosure.  The particular framework for minimizing password discovery is not important, it is the considerations that I believe apply to any such effort: “AuthzN Password-Independent Keys.”

Lights!  Cameras!!  Rolling … Action !!!

Scene 1: The Setup

Late at night in a Pinsk technical high-school computer lab, our intrepid 5p00k5 carry out the following nefarious scheme:

• Collect a very large list of SHA1 hash values for known passwords.  There are dictionaries of these.  For fun, create more from lists of known bad-choice passwords and any other fun ones that the 5p00k5 toss in while gleefully building the collection.
• For seasoning, the team replaces the first  20-bits of some of the hash values with ‘0’ to indicate that this is a cracked hash.  (These are all cracked of course, this just seasons the exploit.)
• As a final bonus, some hashes that have been obtained in various ways but not yet cracked are included to add to the challenge and the verisimilitude.
• Sort (or simply index) the list on the final 100-bits of the 120-bit values and eliminate duplicates.

Scene 2: The “Leak”

• Place the compiled list of several million hash values somewhere that will invite crowd-sourcing of the discovery of the passwords for each of these.
• Each one has a password; the 5p00k5, already know most of them.
• The ones with the first 20 of the 120 bits set to zero will have 2^20 different hashes that collide with them, some of which may actually be hashes of real passwords. At least one is assured because the 5p00k5 already know that one.
• Periodically, report more as having been cracked and set the first 20 bits of those to zero.

Scene 3: The “Confirmation”

• The 5p00k5 can now sit back and watch the confirmation of their exploit.
• Everywhere around the world, operators of services and sites that use unsalted SHA1 digests as password authenticators will notice matches with digests (“cracked” or not) in their databases.  They will have to assume their system has been hacked.  (It is inevitable that there will be such hits, considering the number of bad and already-known, and easily-cracked passwords that are being used everywhere.)
• In addition, individual users will determine, via services that are available for this purpose, that a password of theirs will match one of the ones (“cracked”-form or not) in the leaked list.
• Compounding the insult, hackers everywhere fire up their hot-shot graphics processors and crowd-source the brute force discover of passwords that correspond to the posted hash values.  These passwords are posted in a growing blizzard of shame.
• Thousands of folks start changing their passwords on any service that announces that some of the “leaked” hash values match some of their accounts.
• The dramatic moment yet to be reported is when someone changes their password and immediately discovers that the new SHA1 digest value is already in the “leaked” list.  (This is bound to happen.  It is noticing that it has happened that may take time.)

Intermission

To demonstrate that the hashes are from real accounts somewhere, the associated user identification also needs to be published.  One trusts that does not happen. 

In the absence of that, notice that it doesn’t matter how few SHA1 digests match those for accounts in anyone’s system.  It is the matches that receive all of the attention.  And matches are guaranteed in a system with millions of accounts.  This is what people notice and it is the credulity on which the exploit relies.

So, is it 5p00k5 or is it real?  Is it Memorex or is it Ella?  Is it margarine or butter?

Whatever the fact of the matter, it is necessary to act as if there really is a hack and the SHA1 digest values are for real accounts that have been hacked.  It is folly to do otherwise.  The 5p00k5 win.

So long as account identifications are not disclosed, it is not possible to tell the difference between a real hack and the work of 5p00k5.

It is a vivid demonstration that we have been casual for far too long, using theatrical gestures with no security and safety foundation.

Now what?

You can see that some of the syndicated Planet Apache articles are not so limited. 

I can also see why some projects might want to have their own planet in the system that has ASF as the star.  At the moment the only appropriate active planet is Committers.  It might do for Apache OpenOffice to have its own (with ODF Toolkit as its primary satellite?).

But for now, we’ll do it this way.  This will provide an informal avenue and association separate from the more-formal Apache OpenOffice blog and its project-management voice.

This is why I needed to figure out how WordPress provides separate RSS feeds for a blog’s categories.  This post is by way of fair warning.

A Column about ZIP — Slaw

(We owe this sudden bust-out into blogging to the fact that my de.licio.us tag button just doesn’t work and my Blog This one works just fine.  Nya, Nya, Nya.)

• John Blossom, in a 2011-03-24 post, “Will Fluid Flow to Reality?”, sees Fluidinfo having to contend with Google.  The comparison is with the Google properties served by APIs.  While there might be a collision of some sort, I don’t think the comparison is apt.  For one thing, we don’t have to take any active part in Google’s aggregation of data and delivery of search results.  With Fluidinfo, we (or software that acts for us) are the submitters of “semantic” metadata and we are the ones that have to identify the kinds of our items and what other Fluidinfo-represented entity the data items are related/relevant to.  In effect, we are the curators and the aggregation belongs to Fluidinfo, not us. 
• If this is the semantic web, it means we are its crowd-source curators and I don’t see how that scales from Fluidinfo as we know of it today.  We have other things to do in our lives and having someone else derive the semantic web is confounded by fine-grained nature of Fluidinfo and the permissions model at that level.  On the other hand, it is shocking that Fluidinfo structures aren’t seen as easily related to RDF.
• In passing, it is clear that Fluidinfo is not a Facebook competitor either.  Certainly Facebook holds relationships and connections for us, the appeal being their relevance to our personal social interests.  But we don’t have to curate much and the privacy/permissions model is Facebook’s, not ours.  Nowhere does Facebook expose anything as fine-grained as the individual attribute and relationship items of Fluidinfo.  That’s all behind the curtain. 
• I can imagine a Facebook-like system having a Fluidinfo-style database somewhere underneath, but it would not be exposed to its users and I doubt the service would every expose a Fluidinfo API.  I can’t conceive of the property rights delegating or elevating in any way for that to work.
• Dylan Love, in his 2011-03-14 summary of Tim O’Reilly’s SXSW interview by Jason Calconis, suggests that Fluidinfo is comparable to Wikipedia on a larger scale. 
• I disagree that Fluidinfo is like a wiki[pedia], even though the developers imagine differently.  For one thing, Fluidinfo entries are editable by anyone.  You can only edit your own little bits.  In addition, wikis are versioned and the curation of a wiki involves managing the structure and providing a means of navigation and discovery.  The dance is very different as is the support to discovery of places to put things, other places to hook up to.  Fluidinfo’s not even a wiki based on micro-content, since the micro-level permissions would be a nightmare.
• Although it is not clear how well the FluidDB handles unstructured text in sizable chunks, I can imagine Fluidinfo usage for a kind of FriendFeed that has annotations of others, some private to others, some shared with the originator of an object, some shared with those who can see that object, etc.

The Fluidinfo team is rapidly constructing demonstrations of what can be done, with impressive results.  There are new tools that might help one understand the kinds of structures being used, such as the Fluidinfo Explorer.

It appears that, as much as I am wary of Fluidinfo, the way to wrestle this elephant to the ground is to try it.  Fluidinfo … Like a Virgin?

This appears to be an overwhelming abundance.  It will certainly figure in what there might be available at the C/C++ level, although the connection of UI Automation with Windows Presentation Foundation may not fit my ideas for simple cases.

There is clearly a great deal that a Microsoft-oriented developer can learn here.   I can start with that.  This is certainly a step above the fussing around about accessibility provisions in markup languages.   I’m not sure about the open-sourced add-ins for Microsoft Office.  Those seem to be at a different level.   All is welcome.

The announcement of “O’Reilly’s new API” and a contest for its creative use just caught my eye on Twitter.

I’m intrigued by the broad sweep of the tagline: Openly writable shared metadata for everything.  Everything!

I must know more.  Mustn’t you?

The use of ligatures fi and fl (U+FB01 and U+FB02) is amusing and slick, as is the home page for the fluidinfo site.  Everything is an invitation to try it and like it.  It is all very cleanly done.

Dancing Around the Unsaid

Very clean, except for one little thing.  What exactly is it?  I see that it aggregates metadata and relies on domain names for decentralized establishment of authorities.  But exactly what is it I am using, participating in, contributing to, and jointed-at-the-hip with when I sign up to play?

The About page is a clue: “Fluidinfo is an online information storage and search platform”

Hmm, it is a thing: a storage and search platform.   There’s all this puffing about openly-writable metadata, sharing, and modern writable APIs.  But at the heart there’s this platform thing.  Is it just one platform thing or is this some federative arrangement that the API supports?

How Open is Open for fluidinfo?

The first step towards finding out what (and whose) game you’ll be playing in is to go to the sign-up form.  Here’s the first clue that something is not straight about this, a kind of non-disclosing disclosure:

• “Sign up for API access to Fluidinfo.”
  Very well, we are registering for access to Fluidinfo through a furnished API.  It is not so much about the API as it is access to the Fluidinfo platform thingy
• “You’ll need your username and password to access the API.”
  Well, all right. I think of Fluidinfo as what is being accessed, and an (here: “the”) API is the means of access or maybe the entry-point.  I have come to think of an API as an interface behind which there may be any variety of services that accept that interface, but it appears we don’t mean one quite that high-flown.  Here, we seem to need to think of the specific API by which the Fluidinfo platform is accessed.  They are conceptually joined-at-the-hip.

So, we are accessing a platform thingy, and it belongs to someone.  There are terms of use that we are required to agree to (by at least clicking the box on the sign-up form).

It is clear to me, on arriving at this point, that I am invited to play in someone’s silo.  At this point, I can’t tell if it is an “open-data” silo with an exciting API or is some other kind of fish with an exciting API.

Perhaps the Terms Are Important?

The terms and conditions linked from the sign-up form remove any doubt about what it is we are signing up for.  The definitions at the beginning establish the essentials:

• Company: FluidInfo, Inc., the entity offering a data service.
• API: is how the data service is designated, confirming that we mean access to fluidinfo and “API” is never described or used in any other manner in the terms.
• Licensee: you or I.

Hmm, and I Am Giving Up What?

There are some promises that Licensee (you or I) must make as part of enjoying the use of the service.  These are clearly spelled out.  It is important to know what they are, along with other limitations:

1. The limited license granted to Licensee is “to use the API for the purpose of making procedure calls to Company servers for storage, retrieval and/or manipulation of Licensee’s information (the ‘Content’).”  Note that this is not anyone else’s Content, only our own.
       (I think this is an unintended consequence of “retrieval” being lumped here, unless broader retrieval is by other than “the API.”)
2. Licensee (that’s you and I) “will not patent anything that relates to, or builds up, extends, supplements, is based on or surrounds any aspect of any portion of the API.” Umm, well, API is the service offered by Company, so that might make some sense, although something broader may be reached for here.
3. But not to worry.  If we do happen to breach (2), we are also granting Company all rights to the patent.  Well, at least it is not an exclusive grant, although it appears that Company can do whatever they want with it, including giving away licenses.
4. It gets better.  The API (here it is quite murky what the scope is here) and all intellectual property rights in it are the sole and exclusive property of Company and if, by some strange circumstance Licensee may have any interest in such intellectual property, it is hereby transferred and assigned to Company. 
5. I don’t think I need to look any deeper, but scrolling down to the item on Government Use I noticed this interesting statement: “The API is a ‘commercial item,’ ‘commercial computer software’ and ‘commercial computer software documentation.’ ”

I can’t recall any license for use of a free service, however silo-enclosed, that embroiled me in an intellectual property transfer.   This has me worry that I haven’t been reading the fine print in other click-through licenses closely enough. 

One advantage of the fluidinfo terms and conditions is that they are brief and to the point (though I still have trouble parsing “API”), so I could get the gist of them without lapsing into a coma first.  How’s that for you?

One Last Word

Looking at the Fluid Info API Documentation, especially the HTTP API, it is clear that others could offer the same interface at their own access point (e.g., by specifying a different API host with whatever ports it will allow).  This makes terms and conditions with respect to the “API” rather curious, and it also raises questions, for me, about what the reach of intellectual property claims with respect to the “API” is intended to be.

I’ve been following smatterings of the HTML5 debates on what is in, what is out, and who appointed who God.

Shelley Powers shines a light on much of this from a refreshing observation point that tends to ask the bigger questions that should be informing the minutiae debates that appear to consume the undertaking.  (I need to keep her in mind when I engage in my own particular minutiae crusades.)

I take particular note of matters related to accessibility and provisions in document formats, such as HTML5, that are proposed to facilitate accessibility with the intention of promoting universal access.

Something strikes me as particularly discordant when I inspect the W3C poll results on provisions for a summary attribute (@summary in W3C-speak).  It is difficult to know the full context, but there is still much to be noticed:

1. The summary attribute already exists in HTML 4.01, it is not deprecated, and it is summarized as expressing “purpose/structure for speech output.”  According to the specification, “This attribute provides a summary of the table’s purpose and structure for user agents rendering to non-visual media such as speech and Braille.”  The attribute is also singled out in the actions that user agents (i.e., browsers) may carry out when rendering a table and what authors should do to provide for that eventuality: “[User agents may] make the table summary available to the user.  Authors should provide a summary of of a table’s content and structure so that people using non-visual user agents may better understand it.”
2. “What’s wrong with that?,” you might ask.  Me too.
3. Confessing that I never knew anything about this attribute before this moment, I also note that there is a move to obsolete this attribute in HTML5, along with the apparently-ill-fated longdesc attribute.   Now, it is sometimes good to deprecate features that are under-used, contradictory, cumbersome, not generally supported, and perhaps supplanted by superior alternatives.  Simplification can be important improvement.   Considering how we can’t know the diversity with which HTML 4.0 is applied around the world, it is not clear how we can do without this and have the result it is intended for. 
4. There are some very strange objections with regard to the questionnaire (which is still running).   Some have to do with it being unnecessary on some tables, some only considering tables that are systematic enumerations of some sort of data, some having to do with the desirability of the summary generally, some assuming that @summary information is excluded from availability to sighted users, etc.  There seem to be too many false contrasts.
5. There are also weird variations in the question that may not reflect what the proposal actually is.  I note, in the first comment by Laura Carlson that the proposal is to restore @summary from HTML5 +obsolete status to being a first-class optional attribute of <table> elements.
6. I could go into the minutiae and express my own preferences, but that is not where I want to go with this. 
7. I do note that groups with specific concerns about accessibility have requested retention of @summary and it is passing strange that anyone else considers themselves an authority in how misguided they are in these requests.
8. It can be (and is) argued that tearing out things that are not considered harmful and are already established in earlier HTML versions may be a waste of energy when there are more pressing matters to address.

I think we should be careful.  I see value in considering that there may be creative applications of accessibility provisions that will serve a broad community and be less brittle about what is associated with disabilities and what is not.  It is also valuable to have more fluidity and dynamic choice in who relies on what provision when.   (I can imagine wanting to hear a summary on my mobile phone before attempting to access the details of a table, for one instant-design example.) 

But What Are We Doing Here?

Most of all, I am struck by the degree to which accessibility is presented as something provided on behalf of consumers of web content.  I’ve found no consideration of how persons with disabilities might choose to author web content on behalf of themselves and others, with or without comparable disabilities, with or without assistive appliances of one sort or another. 

Shouldn’t we be drawing on the expertise of such individuals?  Shouldn’t we be getting out of the way of their full expression and the full variety of approaches that might be desired in accessing web content?

Or is some sort of technocratic noblesse oblige arrogance at play here?

[I suppose that if Florian Mueller allowed comments on his blog, it would invite endless rivers of nonsense, so I will rely on my own little stream of nonsense for this.]

There is an interesting discussion going on concerning how Google adapted some header files used with Linux after removing the GNU Public License (GPL) notice and all descriptive text.  The question is whether or not that constitutes a copyright violation.

Informally, one would think so.   FLOSS proponents would be inclined to conclude that to be the case without any further consideration.  Florian quotes Linus Torvalds, who emphatically thinks so (as a general matter with respect to the particular headers it seems, not just with respect to what Google may have done). 

Florian goes on to extrapolate consequences of a determination that the GPL has been infringed and what compliance would require, such as open-sourcing of the Adobe Flash implementation in Android.  I don’t think that follows as the necessary cure, but I don’t want to pursue that analysis regardless.  There are other roads to be explored before that becomes a consideration.

Not being a lawyer, I think I can safely pontificate on this matter.  My understanding is that of a US Copyright law junkie.

The key take-away: Copyright law is such that the determination of infringement, and the nature and degree of the infringement is determined in court.  For example, the fair-use doctrine, as embodied in the law, is applied by courts, not the public, and the copyright law says what the court shall take into consideration but it doesn’t say how the judge will weigh those factors. 

I’d be surprised if Google would present a fair-use defense in this case, but fair use is not the only place that considerable judicial discretion is applicable to whatever facts of the matter are determined in court.

Consider these important factors:

1. Not everything in a copyrighted work is subject to that copyright.  A copyright notice might be indiscriminate when placed on a work, but it still does not apply to non-copyrightable matter in that work and it definitely does not apply to [derivative] content to which the copyright of another applies.  (Google is charged with removing a notice, not adding one, but in some sense the copyrighted subject matter would still need to be determined.)
     
2. Some expressions are inherently not copyrightable, whether or not expressed within a copyrighted work.  It looks like Google convinced itself that it could go down this road.  It is up to a court to decide if what is left in Google’s extractions is such material or not.  Keep in mind that it is generally considered to be the case that APIs are not themselves copyrighted (that is, using them is not a copyright infringement), and there’s a general understanding that one cannot copyright language.  I am oversimplifying.  The determination happens in court, if it goes that far.
      
3. Utilitarian necessity may enter into the situation.  I have not heard any discussion of this in recent years but it is not clear that the doctrine has been pre-empted by revisions made to the US Copyright law starting in 1976.  The idea is that if there is really only one way to express something, that expression is not subject to copyright.  This is an interesting matter with respect to software, and especially with the use of Application Programming Interfaces (APIs) and the headers/declarations that are essential to making use of those APIs.  To the degree that there is something essential in order to express the use of an API, it may be argued that copyright does not apply.  (There, one must resort to trade secrets and contracts, if not patents, to achieve protection and none of that works for FLOSS.  Sorry.)

Of course, I have no idea how a court would come down on the situation with respect to Google’s Android software, nor do I have access to the facts of the matter.  I am a distant, Monday-morning bystander just like the rest of us. 

I would not have make the bet that Google seems to have made, even if there turns out to be no case for infringement.   There had to be a better way, and it would have been more trustworthy to have pursued it and then been absolutely clear on the provenance of material that others are expected to rely on in supporting and enriching the Android platform for mutual fun and profit.  The problem is that Google’s gamble, if there is one, has consequences for others.  Can you spell “evil?”  Of course you can.

PS: If you really enjoy making your head hurt, consider that the way header files in C/C++ programs are effectively used is by mechanically copying them into the programs which include them and then compiling that program as a single unit.  Although the programmer may never see this happening (and some software “pre-compiles” headers), this may tempt one to invoke notions of secondary infringement, etc.  Please don’t go down that road, no matter how much you might dream that any program that runs atop Linux shall be GPL licensed.  Enter that serpent nest of unintended consequences at your own peril.

[update 2011-03-18T20:43 I cleaned up a sentence with a repair that Florian suggested.]

I have been musing about what it takes to architect a family of document-related utilities in ways that address accessibility and internationalization/localization from the outset in a scalable, reusable way.

The hard case for me at the moment is in terms of command-line CUA-level (text-screen) tools.  It means dealing with the ability to have accessibility aids tied into the interactive behavior of software so that it can be operated by people with disabilities, whether programmers, system administrators, or anyone else performing routine tasks of some sort.   In the context of office-productivity tools, this means accessibility for authors, not just for consumers and translators of authored material.

I have no expertise in this area, and I understand the kinds of tools more from the perspective of a typical user (though I use my screen magnifier from time to time).  So this is a daunting prospect because I am not a member of a special-needs target community (although as an American I qualify as language-impaired so I can see provision of that sort of assistance more clearly).

These STAMP and DAISY add-ins for Microsoft Office, now in beta, are developed as open-source packages on SourceForge gives me something to examine and to look at.   I’ve not quite figured out where I can safely install beta software, unless I bring up a few more Virtual Machines, now that I have that capacity on my new developer-centric desktop system.

There are also useful tools being developed around ODF (or LibreOffice, I am never quite sure what the specifics of described ODF-ness are) and it would be interesting to see what sort of harmonization is applicable, if any.

Time to check out the SourceForge sites:

• STAMP: PowerPoint Subtitling Add-In (SourceForge, Microsoft Public License)
• Open XML to DAISY XML Translator (SourceForge, BSD License)

[2011-03-17 Side Note: I have been keeping as many irons in the air as possible, but this means the recovery of my now-dormant blogs from Blogger FTP to hosted Movable Type is taking a back seat to other more-pressing activities.  I am using this WordPress.com incarnation of my Live Hideout as a parking place for posts I’d rather be making elsewhere and I may well cross-post them when I have caught up with my ambitions at least that much.]

Related articles