The templating system works the same as class inheritance.
The HomePage class uses the HomePage.ss template if available, otherwise it uses the template of the base class, in most cases Page.ss. The same is true for Layouts. So, yes: The template with the same name as the Class takes precedence over other templates. It does so automatically, you don't need to change/add anything in your code for that.
If you're using layouts, your page will always be composed of the main template (in the template folder) and a sub-template from the layout folder.
Example for templates/Page.ss:
<html>
<head>
<title>Example</title>
</head>
<body>
<div id="Content">
$Layout
</div>
</body>
</html>
Example for templates/Layout/Page.ss:
<p>
Hello from Page
</p>
Example for templates/Layout/HomePage.ss:
<p><strong>
Hello from the HomePage
</strong></p>
Calling a page of type HomePage will result in the following output:
<html>
<head>
<title>Example</title>
</head>
<body>
<div id="Content">
<p><strong>
Hello from the HomePage
</strong></p>
</div>
</body>
</html>
And Page will output the following html
<html>
<head>
<title>Example</title>
</head>
<body>
<div id="Content">
<p>
Hello from Page
</p>
</div>
</body>
</html>
You're also confusing classes with templates. The code in mysite/code/PageHome.php is the class definition. The files ending in .ss are template files.
This is also covered in Tutorial 1: http://doc.silverstripe.org/doku.php?id=tutorial:1-building-a-basic-site#using_a_subtemplate