See Mordechai Ben-Ari, Mathematical Logic for Computer Science (3rd ed - 2012), page 33 :
The method of semantic tableaux is an efficient decision procedure for satisfiability
(and by duality validity) in propositional logic.
The principle behind semantic tableaux is very simple: search for a model (satisfying
interpretation) by decomposing the formula into sets of atoms [e.g. propositional letters : $p,q,\ldots$] and negations of atoms. It is easy to check if there is an interpretation for each set: a set of atoms and negations of atoms is satisfiable iff the set does not contain an atom $p$ and its negation $¬p$. The formula is satisfiable iff one of these sets is satisfiable.
For each formula, every step is uniquely defined, because you have to decompose the formula according to the principal connective.
See page 33 :
A literal is an atom or the negation of an atom. An atom is a positive literal and the negation of an atom is a negative literal. For any atom $p, \{ p,¬p \}$ is
a complementary pair of literals.
Let :
$$A = p ∧ (¬q ∨¬p).$$
The principal operator of $A$ is conjunction, so [by truth-tables for connectives; see page 16] $v_I (A) = T$ if and only if both $v_I (p) = T$ and $v_I (¬q ∨¬p) = T$.
The principal operator of $¬q ∨¬p$ is disjunction, so $v_I (¬q ∨¬p) = T$ if and
only if either $v_I (¬q) = T$ or $v_I (¬p) = T$.
Thus we have to apply the procedure to $A$ with the goal of verifying if the formuala $A$ is satisfiable or not.
The procedure will always end, because a formula is a finite string of symbols, with only a finite number of occurrences of connectives.
At every step we decompose the final a formula of a branch according to the principal connective applying the above rules.
If the formula $B$ has as principal connective the conjunction, it is $B_1 \land B_2$. Thus, according to the rules for evaluating the connectives, in order to satisfy $B$ we have that both $B_1$ and $B_2$ must be true. Thus, we add a new node to the branch with both $B_1$ and $B_2$.
If the formula $B$ has as principal connective the disjunction, it is $B_1 \lor B_2$. Thus, according to the rules for evaluating the connectives, in order to satisfy $B$ we have that at least one of $B_1$ and $B_2$ must be true. Thus, we branch, one for each possibility.
Part A
Thus, for :
$p ∧ (¬q ∨ ¬p)$
you can only apply the rule for $\land$, because it is the princupal connective.
In the second step :
$p, ¬q ∨ ¬p$
$p$ is atomic, i.e. indecomposable. Thus you can only decompose $¬q ∨ ¬p$, applying the rule for $\lor$.
Part B
You have two formulae; thus you can choose how to start.
First decompose $\lnot p \land \lnot q$ and then $p \lor q$, with the branching.
Thus, the "strategy" is : if you can (as in B) use the "branching" rules at the end, in order to have more "compact" trees.