MLlib, Linear Regression Test

Source:

Result:
I don't know why i am getting those EOF errors. Maybe some bugs in new Spark 1.4.0 package..?

And this MLlib doesnt care about R square of P-values .. maybe because of speed issue? (So far this is very slow indeed...) Calculating them including MSE will require remapping and re-reducing. I am not sure this is the way it should be, but even if it is, it's another business opportunity I guess. (That is, if i survive from poverty until then)

I will try to find out about getting P-values in MLlib trainer in the next session..

Also, Spark does provide distributed matrix data types:
https://spark.apache.org/docs/latest/mllib-data-types.html#distributed-matrix

I will test them with some simple numerical anlaysis examples.. soon..