For a regular polygon with n sides (n>5), the number of triangles whose vertices are joining non-adjacent vertices of the polygon is
(a)n(n-4)(n-5)
(b)(n-3)(n-4)(n-5)/3
©2(n-3)(n-4)(n-5)
(d)n(n-4)(n-5)/6
Solve it by hand for n=6 and put the values in all the options 
The answer is n(n-4)(n-5).
First let us count all the triangles possible. For this we need to choose any 3 vertices. This can be done in C(n,3) ways. Now from this we have to subtract two values :
i) the number of triangles with exactly 2 adjacent vertices
ii) the number of triangles with all 3 vertices adjacent
Calculation of i)
There are a total of n pairs with adjacent vertices. We can select any pair in n ways. After this, we have to choose any vertex from the remaining (n-4) vertices as we cant choose the chosen vertices as well as the vertices adjacent to them. Hence (i) amounts to n(n-4).
Calculation of (ii)
There are a total of n triplets where all the vertices are adjacent. Hence we can choose a triplet in n ways.
Finally, the answer is C(n,3) - n(n-4) - n.
= n(n-1)(n-2) / 6 - n(n-3)
= (n^3 - 3n^2 + 2n - 6n^2 + 18n) / 6
= (n^3 - 9n^2 + 20n) / 6
= n(n-4)(n-5) / 6
(Sorry for the improper indentations)
2 Likes
however i have given answer but please tell be a detail solution.
Thanks its a nice explanation.