It seems an easy assertion that "communication communicates" when looking at open broadcasts: Like the books you work with, or even forum posts. The spoken word is quite different, and even more so the fewer in number the audience. A blind spot for academia/rationalists is that "people matter", and not in just that trite sounding way, but connection. There is no such connection with the broadcasted word.
These are good points, and hit on a central issue in linguistics/semiotics: namely, whether there's a difference between spoken and written language. I think it's safe to say that yes, there are many differences, some quite obvious; but I think there are also similarities that complicate the matter.
I don't really want to contend with the clarity and/or correctness of Derrida's work, but he makes a provocative point in an essay titled "Signature Event Context." In short, he contends that an implicit graphability, or iterability, subsists in both written and spoken language, and I find this to be a compelling claim:
[The] possibility of extraction and of citational grafting which belongs to the structure of every mark, spoken or written, and which constitutes every mark as writing even before and outside every horizon of semiolinguistic communication; as writing, that is, as a possibility of functioning cut off, at a certain point, from its "original" meaning and from its belonging to a saturable and constraining context. Every sign, linguistic or nonlinguistic, spoken or written (in the usual sense of this opposition), as a small or large unity, can be cited, put between quotation marks; thereby it can break with every given context, and engender infinitely new contexts in an absolutely nonsaturable fashion.
Derrida will refer to this as "arche-writing" - that even spoken language presupposes a written (i.e. citational) origin, even as it breaks away from and defies this origin. As I said, we could go back and forth on Derrida for eternity; but that's what he says, and I think it merits consideration.
Alternatively, someone like Wittgenstein would probably disagree and fall more on your side of the issue; but even Wittgenstein acknowledges that meaning is precarious in spoken contexts, even intimate ones. As the model of rule-following demonstrates, it circulates endlessly through a series of rules that are continuously followed and/or broken, and new rules are always being engendered. But the process always has to presuppose a set of rules that somehow existed prior to the speech act. This seems to mesh nicely (in my opinion) with Derrida's insistence that all marks and utterances act upon the (il)logic of an already-existing written form. In other words, even conversation, or "connection," as you say, relies on the preexistence of communication. A statement, written or spoken, only makes sense if it can be translated. This is why Wittgenstein contends that there can be no such thing as a private language.
I'm not sure what the distinction, or possibly the overlap, between "academia" and "rationalists" is; and I'm not sure I agree with the accusation that people don't matter for academicians (by which I assume you mean scholars in the humanities). I think that people can matter just as much even if we shift our focus from the nodes to the network.
EDIT: if we're thinking, for example, about psychology or psychoanalysis, the immediacy of the environment certainly plays a role; but I wouldn't say this negates the communicational structures on which the psychological relationship is constructed.