The name or title, Church of God, is undeniably the true and proper appellation by which the New Testament church ought to be designated. This is her scriptural and appropriate name. This, and no other title, is given her by divine authority (Isa 62:2 Ac 20:28 Ga 1:13 1Ti 3:15). This name or title, therefore, ought to be adopted and worn to the exclusion of all others.
Full text here.
“But with regard to the concluding item–that I “refused to baptize children, when requested,” I must say it is true. Yet, I never once positively refused without a good reason for it. You would be astonished, brethren, were I to tell you but one half of the extent of the superstition and corruption that prevails among us, on this solemn subject. Some who live at large in the world, without being connected with any Christian denomination, and obviously take no thought for their own souls, though they are in jeopardy every hour, will still come and want their children baptized. And if asked whether they have a scriptural warrant to covenant with God, on part of their child, they will often evade the force of the question.”
This is taken from his recounting of the circumstances surrounding his leaving the German Reformed Congregation in Harrisburg.
“This unity of faith and sentiment some think impossible. We once preached in a certain place, on Christian union, when the minister in charge said to us, he thought the people could never be brought to believe alike. We asked him why not? He replied, because men were so differently constituted, that they could  not believe alike. We then asked him, whether men had not now, precisely the same organism, physically, mentally and morally, which they had eighteen hundred years ago? He allowed they had. We further asked him whether the primitive Christians were not all one in faith and sentiment, as well as one in heart? He replied in the affirmative, but added that they were under the teaching of inspired men. Well, said we, if we teach and inculcate the same doctrine, and the people have the same mental constitution, why then, should they not all have one faith?
Besides, we said, is it not true that all men are alike bound to believe God? And what God requires one man to believe, does He not require all to believe?
Again, said we, how could God consistently require all men everywhere to repent and believe the gospel, and yet create them with such diversified mental constitutions, that they could not understand and believe the gospel alike? The trouble, said we, lies not in the gospel, nor in the minds of the people, but in the teachings of their blind guides. Let but ministers learn to speak as the oracles of God–let them all preach the same truths which were taught by the apostles, and the people will soon come to the unity of the faith in the bonds of peace. But, the texts above quoted, fully settle the point, that Christians are required to be of the same mind, and of the same faith.”
Sermon on Christian Union
“It is contrary to Scripture to divide the church of God into different sects and denominations. This is sufficiently evident from the fact, as I before showed, that the word ecclesia, or church, is never used by the inspired penmen in such a sense, but always as denoting either the whole collective body of the faithful throughout the world, or a distinct congregation of Christians located in some given place. Accordingly, there are individual or particular churches; and those collectively constitute one general or universal church. Beside this division of the church, there is no divine warrant given for any other. Hence, the combination of two or more individual churches into a sect or distinct connection wearing a sectarian name and governed by human laws is highly improper and anti-scriptural.”
From ‘A Brief Scriptural View of the Church of God’ (1885), Chapter One.
Click HERE to find see this document online.