But we don't all understand the same thing from language that doesn't closely follow our agreed rules of language structure. Specifically in our example, not everyone will understand what the mangled translation of my text says.

So this is something to wonder about: why is it that we all speak in a certain way? If we can understand language with unfamiliar structure, why does language follow set patterns? Why don't we all speak any old way we like?

Mind you, I'm not saying I have any answer to this. Because it's actually a major question. You could, for example, observe that people will often break from accepted conventions when speaking or writing and it's hard to even measure the deviation from accepted conventions "in the wild" (as opposed to more formal text or speech). And yet there does seem to be a set of patterns of utterances that we observe when speaking or writing in our respective languages. There are rules, in language, and these rules are observed. Why is that?

Understanding the answer to this question -what makes language, language, rather than random noise- could tremendously boost NLP tasks of all sorts. To be honest, I don't see anyone asking it though. Instead people have become comfortable with the paradox of the random noise (sorry, I insist) generated by computer programs, and treat it as an engineering problem to be solved, rather than an opportunity to understand human language by looking at its difference to machine language (if I may).

>> (please, don't ask anything that starts with "how do you know...?". I know. Let's drop that angle, please!)

I understand if you don't want to spend time discussing this, but I hope you understand that, unless you justify your thinking, you can't expect anyone else to agree with it.

In any case, "how do you know" is the fundamental question you have to ask if you want to advance your knowledge. That's how it all begins- by challengine assumptions (yours or others').

> Specifically in our example, not everyone will understand what the mangled translation of my text says.

Not everyone will understand your hand-made translation, either. Human understanding works by consensus. Someone will be left out of this consensus almost always. It doesn't mean the automated translation is gibberish. If enough people understand the automated translation, then by definition it's not gibberish, because random noise doesn't carry enough information for you to get the idea about Maria, Vassiliki, the hospital or the broken leg.

> I understand if you don't want to spend time discussing this, but I hope you understand that, unless you justify your thinking, you can't expect anyone else to agree with it.

I have justified it. I asked you to drop it because we're running in circles, not because I don't have arguments. Your argument seems solipsistic to me: "but how do you know?" -- I know because it's my experience and because other people agree with me. How do you know your translation is good, anyway? Because it's what you think and because other people agreed with you. Because you managed to successfully -- as assessed by yourself -- communicate the idea you wanted.

The difference is that I can point at the ways that GT fails, in terms of specific grammatical and semantic errors, whereas you can only report some examples of people who agree with you. Not trying to be offensive, but your definition of gibberish is subjective whereas mine is objective and quantifiable (in one word: ungrammaticality).

To go back to an earlier example of learning to solve mathematical problems with machine learning- let's call it machine arithmetic. Say we developed a machine arithmetic system that consistently estimated the sum of 4 + 4 to be 7. That's consistently close to the correct answer, but it's also consistently not the correct answer.

So, to pose yet another question: a system trained to solve maths problems who got it almost right 100% of the time, would be considered pretty rubbish. Why is GT not? It's computing language, not maths, but its computation is still very often completely wrong in specific, measurable ways that really leave no room for subjective interpretation.

> The difference is that I can point at the ways that GT fails, in terms of specific grammatical and semantic errors, whereas you can only report some examples of people who agree with you

What matters for the purpose of human communication is the latter, not the former. Success at communication trumps your grammatical errors (where beauty is not concerned, of course. If you're going to argue automated translation produces ugly results, then you'll hear no argument from me!).

> Not trying to be offensive, but your definition of gibberish is subjective

You're not offensive; you're arbitrary. I provided you with a dictionary definition of gibberish, while your definition seems to be suspiciously ad hoc. You've essentially defined gibberish as "whatever output automated translation currently produces", which makes this debate pointless.

Math and natural language are different. Again, the standards are different, just as in your example with child's language. No, we do not judge a computer's ability to "translate" the same as we judge a young kid's ability to speak. There's nothing surprising about these two standards being different.

Your example with Math is not particularly relevant. However, since your background seems not to be computer science [1], maybe you're unfamiliar with heuristics: in the context of computer science, a heuristic is a technique where you take an optimization problem (e.g. "finding the minimum of something", "the shortest route to somewhere", etc) which is computationally untractable and provide a "good enough" answer which can be computed. The definition of what is "good enough" is, of course, relevant to the application of a particular heuristic in a specific context. In this sense, for some heuristic and its application, maybe 7 is a "good enough" answer even though the right answer is 8.

[1] This was written before I read in another reply of yours that you claim to be working on a PhD in Machine Learning. Fair enough! But even if you're familiar with heuristics, I think my point stands.

>> What matters for the purpose of human communication is the latter, not the former. Success at communication trumps your grammatical errors (...)

But grammatical errors cause communication to fail. Because they introduce (measurable) noise to the signal, which obviously makes it harder to extract information from it.

Btw, grammaticality is another uncontroversial measure of the performance of NLP systems that are required to generate language (like machine translation does). There is some theory behind the idea that well-formed sentences are inherently more intelligible, that will take a lot more to overturn than "it works for me".

To draw yet another analogy- homeopathy "works" for some people: they feel better when they take its concoctions. That doesn't make homeopathy objectively useful.

>> in the context of computer science, a heuristic is a technique where you take an optimization problem (e.g. "finding the minimum of something", "the shortest route to somewhere", etc) which is computationally untractable and provide a "good enough" answer which can be computed.

Yes, looking at machine translation as a heuristic is a good point. Like I say in other comments, the problem of machine translation has no good definition. That's because it's very difficult to define what is a "good translation" in a formal and principled manner. Hence, we're left with heuristics like "how close this automatic translation is to this human translation" (where the human translation is whatever happens to be available, rather than something chosen for its quality or other attributes).

This is actually exactly what machine translation systems do. They are made to optimise a measure of error over examples of human translation. So GT is actually designed to do what you say is not important, produce grammatically and syntactically correct language, like its examples are.

In that, it fails. It may achieve your goal of "communication" but that is not really a heuristic, it's more of a quality that you, personally (and obviously other people too) assign some value to. But it's far from a universally accepted measure of machine translation quality. You might say that "communication" is a business goal whereas matching human translations is an engineering goal.

As a P.S., I'm not "claiming" to be working on a PhD. The information is in my profile with links to my linkedin and github :)

> But grammatical errors cause communication to fail. Because they introduce (measurable) noise to the signal, which obviously makes it harder to extract information from it.

But when communication doesn't fail, grammatical errors don't matter that much. Because communication success is its own evidence (cue your "but how do you know [...]?" -- please don't. We've been down that road and your argument is unconvincing and solipsistic).

You say grammar correctness is another uncontroversial measure of NLP, but nobody was arguing against this. Of course it is. You're shifting the goalpost. Grammar matters. You can tell when something is ungrammatical but whether this affects communication is another issue. It's also a matter of degree: something can be so ungrammatical it conveys no meaning; and something can be ungrammatical enough to sound "bad", but still convey its intended meaning.

> Hence, we're left with heuristics like "how close this automatic translation is to this human translation"

That's one heuristic, sure, and a pretty good one! Not the only one, though. In any case, you missed the point that your Math example was not very good, because it ignored that in a heuristic, what you proposed as a bad "translation" could actually be a good one. In the realm of human language translations, it's never a case of 4+4=8.

> As a P.S., I'm not "claiming" to be working on a PhD. The information is in my profile with links to my linkedin and github :)

My mistake then! No offense intended.

PS: one more thing: I've looked at your comments history and I actually agree with your skepticism about AI. I think it's overhyped, especially by people with no background and no formal understanding about it -- like Eliezer, who you rightfully mock in your profile, and many other Prophets of the AI Apocalypse (or AI Utopia, depending on their mindset). So consider me a skeptic as well. We're more in agreement that it seems! I just don't think the kind of translation we're talking about is anywhere close to General AI or "understanding" of any form, and I can get excited about a "limited breakthrough" even though it's not actually about AI :)

Well, alright then. Let's agree to disagree so I won't say anything else. This thread's text is already pushing very aggressively towards the right side of my browser window anyway.

>> No offense intended.

... and no offense taken, absolutely.

We'll probably get the chance to argue again in the future, anyway, given that you're interested in the same subjects as I am :)