In the book Reminiscences of a Nonagenarian (Huse & Co., Newburyport, 1879) Sarah Emery relates her memories of this first textile mill: "The year I was seven years old the first incorporated woolen mill in Massachusetts was established at the falls on the river Parker, in the Parish of Byfield in Newbury. The erection of this mill created a great sensation throughout the whole region. People visited it from far and near. Ten cents was charged as an admittance fee. That first winter sleighing parties came from all the adjacent towns, and as distant as Hampstead and Derry, in New Hampshire. Row after row of sleighs passed over Crane-neck hill, enlivening the bright, cold days by the joyous tones of their merry bells. Never shall I forget the awe with which I entered what then appeared the vast and imposing edifice. The huge drums that carried the bands on the lower floor, coupled with the novel noise and hum increased this awe; but when I reached the second floor where picking, carding, spinning and weaving were in progress my amazement became complete. The machinery, with the exception of the looms, was driven by water power, the weaving was by hand. Most of the operatives were males, a few young girls being employed in splicing rolls."

It has been argued that this was the first woolen mill in the United States although a woolen manufactory with some machinery operated by water power was established in Hartford in 1786. The technology for water-powered woolen and cotton mills was developed in the 18th century in England, but the English did everything within their power to prevent the machinery from being exported to North America. The early history of the industrial revolution in North America is one of not only inventive genius, but also one of spies and industrial espionage.

In March of 1793, Arthur and John Scholfield sailed from Liverpool, England for America. They arrived in Boston in May and introduced themselves to Rev. Morse, the minister of the First Congregational Church in Charlestown. They told Rev. Jedediah Morse that they were experts in the English woolen manufacturing business and wished to start a business in the United States. Rev. Morse was the father of Prof. Samuel F.B. Morse, the inventor of the telegraph, and was a man deeply interested in any technology which would allow our young country to prosper.

Rev. Morse found housing for the Scholfield brothers and soon, in partnership with John Shaw, began to make woolen cloth in a rented house in Charlestown. John Scholfield made the first machinery, a hand-loom and a spinning-jenny, based on his memories of the equipment he had worked with in England. Rev. Morse was impressed with the Scholfield brothers abilities and realized that, with the investment of sufficient money and a good location with water power, the manufacture of cloth could turn into a very successful business. Rev. Morse contacted his friend, William Bartlett, a Newburyport merchant and one of the richest men in all of New England. William Bartlett invited the Scholfields to move to Newburyport and to start a woolen factory with improved machinery. They moved their loom and spinning jenny to Newburyport in December of 1793 and immediately constructed a carding machine which was set up in Lord Timothy Dexter's stable where it was operated by hand.

The machines built by the Scholfield brothers created great excitement in the Newburyport business community and no time was wasted. In January of 1794 thirty merchants and other prominent citizens of the area raised $300,000 and received a charter from the Massachusetts Legislature as "The Proprietors of the Newburyport Woolen Manufactory." By March of 1794 the new woolen company purchased 6 acres at the Falls on the Parker River from Captain Paul Moody. Captain Moody was the father of Paul Moody, a machinist who played an important role in the building of the mills in Lowell. This was the site of the first mill in Newbury and the Moody family had been operating a saw mill and perhaps other mills at this site since 1710.

A large building was designed and was probably completed in October of 1794. The new factory was 100 feet long, 40 feet wide and three stories high, quite a sight at a time when a typical village mill might be 45 feet long, 20 feet wide and only two stories high.

In 1804 the woolen factory was purchased by John Lees for $8,500. John Lees converted the mill to the manufacture of cotton goods. He succeeded in importing the needed machinery from England by having it packed in casks labeled "Hardware" before it was placed on the ship. To avoid suspicion he shipped the drawing and spinning-frames in a separate ship.

Sarah Emery goes on to tell us: "the establishment of this factory brought quite a revolution in the domestic manufactures of the neighbor hood. For some time previous, in most families hand carding had been discontinued, the wool having been sent to be converted into rolls to the clothier mills of Mr. Ben. Pearson or Mr. Samuel Dummer. Lees & Taylor made arrangements by which this family carding could be done at their factory both cheaper and better than at the smaller mills. The introduction of cotton opened a new channel of industry. The weaving was still performed by hand: as the business increased this loom power was not sufficient to supply the demand for cloths. Their goods consisted of heavy tickings and a lighter cloth of blue and white striped or checked, suitable for men's and boys's summer wear, aprons, &c. The tickings were woven by men on the looms at the factory, but much of the lighter stuffs were taken into families and woven on the common house loom. The yarns were spun and dyed at the factory; these could be purchased there, and in lieu of the hitherto universal linen and tow, cotton began to be mixed with flax or woven alone. ... This cotton spinning brought a new occupation to the place. Being prior to Whitney's invention of the cotton gin, the material came just as it had been gathered from the field, and many of the females in the neighborhood of the factory were employed to separate the seed from the cotton. For years one rarely entered a farm house in the vicinity without finding one or more of the inmates busy picking cotton."

The mill was converted back to a woolen mill in 1839. In 1860 it was rebuilt after a major fire and continued in operation until a fire in 1932. All that remains today is the mill dam just west of the Central Street bridge over the Parker River and the foundation of one of the mill buildings which can be seen on the east side of Central Street..