First, when I use the word "Church," I refer to Christ's bride, those who are or will be saved, and not a place where people go to meet to worship.
Now, using typology in the Bible can help us find answers to the above questions. For review, briefly, typology is seeing the spiritual realm via the temporal.
Let's start with the people of Israel -- God's "chosen people." God called them out from among all other nations of people to be His people (Deut 7:6). While God temporily saved the people of Israel out from bondage of Egypt as a "type" (shadow) of spiritual salvation and separating a people (His children, the Church) unto Himself, later on, many of those individuals who were brought out of Egypt perished in their sins (Num 16; Num 14:22-34; Num 21:4-9). How could this be if they were part of the group called "God's chosen people"?
In Jer 5:25-31, God declares that among His people are found wicked men and false prophets; but how can they be such if they are His people? The Lord goes on further to say in Jer 23:3 that He will gather the remnant of the people of Israel who He scattered to other countries. Why only the remnant? Are they not all God's chosen people?
During His time here on earth, much of Christ's ministry was the condemnation of the Jewish religious people of the day (John 8:44; Matt 23:27). They called themselves God's chosen people, their genealogical father being Abraham; but Christ condemned them; and yet, there were a few around Him of the same Jewish chosen people that had a God-given faith in Him (Matt 16:16; John 11:24-27). How is that possible?
As you can see with these examples, God deals with these two groups within the "people of God" separately, one being false professors of religion, the others being actual (true) followers of God. But what about the heathen as a group? If you look at it high-level, most of who Scripture addresses are the people of God in general, and the heathen are not really dealt with by God or even Christ, other than in their relation to God's people -- all of the Old Testament is generally directed toward God's chosen people, Christ never really dealt with the heathen authorities, and the New Testament is essentially written to Churches in various locations. And so, it is evident that there are three groups of people in the Bible -- the two divisions of God's chosen people, and the heathen -- and not just two groups -- the saved vs. the not saved, Christians vs. non-Christians (although those delineations do exist).
This is shown to be true in the New Testament, where Paul states in Rom 9:6 that "...they are not all Israel [spiritual], which are of Israel [temporal]." And he further describes who are the true seed of Abraham (spiritual Israel) in Gal 3:16,29: "Now to Abraham and his seed were the promises made. He saith not, And to seeds, as of many; but as of one, And to thy seed, which is Christ....And if ye be Christ's, then are ye Abraham's seed [spiritual Israel], and heirs according to the promise." Now, Abraham had his physical seed, which were Isaac and Ishmael -- both the seed of Abraham, but yet one not the spiritual seed, as Ishmael was not of the line chosen by God. The same was with Jacob and Esau. Similarly, the Jewish people at Christ's time were of the temporal seed of Abraham, considered by themselves to be God's chosen people (from lineage); but the religious leaders of the day weren't of the spiritual seed, whereas the apostles were -- both groups of people known as "God's chosen people" from Abraham, and yet only some of them being truly (spiritually) God's chosen people.
As previously noted, these temporal conventions are "types" or shadows of spiritual realities. Spiritually, the heathen are still the heathen; but the people of Israel, those called by Jehovah's name, were "type" for those who are called by Christ's name (this is not an argument for dispensationalism or otherwise -- it's simply type and antitype). Now, since there were those in Israel who weren't godly and those who truly were, the same applies in the spiritual realm, with those who are not Christians even though they call themselves such, and those who actually are Christians. Thus, there are those who are false professors of Christianity, and those who are true Christians (ie. the true Church).
Understanding this then begs the question: how do we know who is part of the true Church? The Bible declares we know them by their doctrine (Rom 6:17; 1 Tim 1:9-11) and their fruit (Matt 7:15-20). Not everyone who claims the name of Christ is saved (Matt 7:21-23), and so having the truth is important, in beliefs and how one lives his life.
May the Lord always give us eyes to see and ears to hear His Word, may He grant us a love of the truth (2 Thess 2:10), and may He grow us in the knowledge and understanding of who He is.